Package it.unimi.dsi.io

Class FileLinesByteArrayIterable

java.lang.Object

it.unimi.dsi.io.FileLinesByteArrayIterable

All Implemented Interfaces:

Size64, Iterable<byte[]>

public class FileLinesByteArrayIterable
extends Object
implements Iterable<byte[]>, Size64

A wrapper exhibiting the lines of a file as an Iterable of byte arrays.

An instance of this class makes it possible to access the lines of a file as an Iterable of byte arrays. Reading is performed using FastBufferedInputStream.readLine(byte[], EnumSet), and follows the rules defined therein. No decoding is performed.

The result of a call to iterator() can be used to scan the file; each call will open an independent input stream. The returned iterator type (FileLinesIterator) is Closeable, and should be closed after usage. Exhausted iterators, however, will be closed automagically.

Using a suitable constructor it is possible to specify a decompression class, which must extend InputStream and provide a constructor accepting an InputStream (e.g., GZIPInputStream if the file is compressed in gzip format).

Convenience static methods makes it possible to build on the fly an iterator over an input stream using the same conventions.

This class implements size64(), which will return the number of lines of the file, computed with a full scan at the first invocation. However, it is also possible to specify at construction time the number of lines in the file to skip the first scan. It is responsibility of the caller that the specified size and the actual number of lines in the file do match.

Since:

2.6.17

Author:

Sebastiano Vigna

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final class	FileLinesByteArrayIterable.FileLinesIterator	An iterator over the lines of a FileLinesByteArrayIterable.

Constructor Summary

Constructors

Constructor	Description
FileLinesByteArrayIterable(String filename)	Creates a file-lines byte-array iterable for the specified filename.
FileLinesByteArrayIterable(String filename, long size)	Creates a file-lines byte-array iterable for the specified filename and size.
FileLinesByteArrayIterable(String filename, long size, Class<? extends InputStream> decompressor)	Creates a file-lines byte-array iterable for the specified filename and size, optionally assuming that the file is compressed.
FileLinesByteArrayIterable(String filename, long size, EnumSet<FastBufferedInputStream.LineTerminator> terminators)	Creates a file-lines byte-array iterable for the specified filename and size using the given line terminators.
FileLinesByteArrayIterable(String filename, long size, EnumSet<FastBufferedInputStream.LineTerminator> terminators, Class<? extends InputStream> decompressor)	Creates a file-lines byte-array iterable for the specified filename and size using the given line terminators and optionally assuming that the file is compressed.
FileLinesByteArrayIterable(String filename, Class<? extends InputStream> decompressor)	Creates a file-lines byte-array iterable for the specified filename, optionally assuming that the file is compressed.

Method Summary

Modifier and Type	Method	Description
ObjectList<byte[]>	allLines()	Returns all lines as a list.
ObjectBigArrayBigList<byte[]>	allLinesBig()	Returns all lines as a big list.
FileLinesByteArrayIterable.FileLinesIterator	iterator()
static FileLinesByteArrayIterable.FileLinesIterator	iterator(InputStream inputStream)	A convenience method returning a one-off FileLinesByteArrayIterable.FileLinesIterator reading from an input stream.
static FileLinesByteArrayIterable.FileLinesIterator	iterator(InputStream inputStream, Class<? extends InputStream> decompressor)	A convenience method returning a one-off FileLinesByteArrayIterable.FileLinesIterator reading from an input stream.
static FileLinesByteArrayIterable.FileLinesIterator	iterator(InputStream inputStream, Class<? extends InputStream> decompressor, EnumSet<FastBufferedInputStream.LineTerminator> terminators)	A convenience method returning a one-off FileLinesByteArrayIterable.FileLinesIterator reading from an input stream.
long	size64()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Methods inherited from interface it.unimi.dsi.fastutil.Size64

size

Constructor Details

FileLinesByteArrayIterable

public FileLinesByteArrayIterable(String filename)

Creates a file-lines byte-array iterable for the specified filename.

Parameters:

filename - a filename.

FileLinesByteArrayIterable

public FileLinesByteArrayIterable(String filename,
 long size)

Creates a file-lines byte-array iterable for the specified filename and size.

Parameters:

filename - a filename.

size - the number of lines in the file.

FileLinesByteArrayIterable

public FileLinesByteArrayIterable(String filename,
 long size,
 EnumSet<FastBufferedInputStream.LineTerminator> terminators)

Creates a file-lines byte-array iterable for the specified filename and size using the given line terminators.

Parameters:

filename - a filename.

size - the number of lines in the file.

terminators - line terminators for the underlying FastBufferedInputStream.

FileLinesByteArrayIterable

public FileLinesByteArrayIterable(String filename,
 Class<? extends InputStream> decompressor)
                           throws NoSuchMethodException,
SecurityException

Creates a file-lines byte-array iterable for the specified filename, optionally assuming that the file is compressed.

Parameters:

filename - a filename.

decompressor - a class extending InputStream that will be used as a decompressor, or null for no decompression.

Throws:

NoSuchMethodException

SecurityException

FileLinesByteArrayIterable

public FileLinesByteArrayIterable(String filename,
 long size,
 Class<? extends InputStream> decompressor)
                           throws NoSuchMethodException,
SecurityException

Creates a file-lines byte-array iterable for the specified filename and size, optionally assuming that the file is compressed.

Parameters:

filename - a filename.

size - the number of lines in the file.

decompressor - a class extending InputStream that will be used as a decompressor, or null for no decompression.

Throws:

NoSuchMethodException

SecurityException

FileLinesByteArrayIterable

public FileLinesByteArrayIterable(String filename,
 long size,
 EnumSet<FastBufferedInputStream.LineTerminator> terminators,
 Class<? extends InputStream> decompressor)
                           throws NoSuchMethodException,
SecurityException

Creates a file-lines byte-array iterable for the specified filename and size using the given line terminators and optionally assuming that the file is compressed.

Parameters:

filename - a filename.

size - the number of lines in the file.

terminators - line terminators for the underlying FastBufferedInputStream.

decompressor - a class extending InputStream that will be used as a decompressor, or null for no decompression.

Throws:

NoSuchMethodException

SecurityException

Method Details

iterator

public FileLinesByteArrayIterable.FileLinesIterator iterator()

Specified by:

iterator in interface Iterable<byte[]>

iterator

public static FileLinesByteArrayIterable.FileLinesIterator iterator(InputStream inputStream)

A convenience method returning a one-off FileLinesByteArrayIterable.FileLinesIterator reading from an input stream.

Parameters:

inputStream - an input stream.

Returns:

an iterator returning the lines contained in the provided input stream.

iterator

public static FileLinesByteArrayIterable.FileLinesIterator iterator(InputStream inputStream,
 Class<? extends InputStream> decompressor)

A convenience method returning a one-off FileLinesByteArrayIterable.FileLinesIterator reading from an input stream.

Parameters:

inputStream - an input stream.

decompressor - a class extending InputStream that will be used as a decompressor, or null for no decompression.

Returns:

an iterator returning the lines contained in the provided input stream.

iterator

public static FileLinesByteArrayIterable.FileLinesIterator iterator(InputStream inputStream,
 Class<? extends InputStream> decompressor,
 EnumSet<FastBufferedInputStream.LineTerminator> terminators)

A convenience method returning a one-off FileLinesByteArrayIterable.FileLinesIterator reading from an input stream.

Parameters:

inputStream - an input stream.

terminators - line terminators for the underlying FastBufferedInputStream.

decompressor - a class extending InputStream that will be used as a decompressor, or null for no decompression.

Returns:

an iterator returning the lines contained in the provided input stream.

size64

public long size64()

Specified by:

size64 in interface Size64

allLines

public ObjectList<byte[]> allLines()

Returns all lines as a list.

Returns:

all lines of the file wrapped by this file-lines byte-array iterable.

See Also:

• allLinesBig()

Implementation Specification:

This method iterates over the lines of the file and accumulates the resulting byte arrays in a standard list. Thus, it will throw an exception on files with more than Integer.MAX_VALUE lines.

allLinesBig

public ObjectBigArrayBigList<byte[]> allLinesBig()

Returns all lines as a big list.

Returns:

all lines of the file wrapped by this file-lines byte-array iterable.

See Also:

• allLines()

Implementation Specification:

This method iterates over the lines of the file and accumulates the resulting byte arrays. in a big list. Thus, it supports files with more than Integer.MAX_VALUE lines.