Class AbstractStore

java.lang.Object
com.mckoi.store.AbstractStore
All Implemented Interfaces:
Store
Direct Known Subclasses:
JournalledFileStore

public abstract class AbstractStore extends Object implements Store
Provides an abstract implementation of Store. This implements a bin based best-fit recycling algorithm. The store manages a structure that points to bins of freed space of specific sizes. When an allocation is requested the structure is searched for the first bin that contains an area that best fits the size requested.

Provided the derived class supports safe atomic IO operations, this store is designed for robustness to the level that at no point is the store left in a unworkable (corrupt) state.

Author:
Tobias Downer
  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected static final long
    The offset into the file that the bin area starts.
    protected static final int
     
    protected static final long
    The offset into the file that the data areas start.
    protected static final long
    The offset into the file of the 64 byte fixed area.
    protected long[]
    The free bin list contains 128 entries pointing to the first available block in the bin.
    protected final byte[]
     
    protected static final int
    The magic value.
    protected boolean
    True if this is read-only.
    protected long
    The total amount of allocated space within this store since the store was openned.
    protected long
    A pointer to the wilderness area (the last deleted area in the store), or -1 if there is no wilderness area.
  • Constructor Summary

    Constructors
    Modifier
    Constructor
    Description
    protected
    AbstractStore(boolean read_only)
    Constructs the store.
  • Method Summary

    Modifier and Type
    Method
    Description
    protected void
    checkPointer(long pointer)
    Checks the pointer is valid.
    void
    Closes the store.
    protected void
    coalescArea(long pointer, long size)
    Coalesc one or more areas into a larger area.
    createArea(long size)
    Allocates a block of memory in the store of the specified size and returns an AreaWriter object that can be used to initialize the contents of the area.
    void
    deleteArea(long id)
    Deletes an area that was previously allocated by the 'createArea' method by the area id.
    protected abstract long
    Returns a pointer to the end of the current data area.
    protected long
    expandDataArea(long minimum_size)
    Expands the data area by at least the minimum size given.
    Scans the area list, and any areas that aren't deleted and aren't found in the given ArrayList are returned as leaked areas.
    Returns a List of Long objects that contain a complete list of all areas in the store.
    getArea(long id)
    Returns an object that allows for the contents of an area (represented by the 'id' parameter) to be read.
    protected void
    getAreaHeader(long pointer, long[] header)
    Sets the 'header' array with information from the header of the given pointer.
    Returns an InputStream implementation that allows for the area with the given identifier to be read sequentially.
    getMutableArea(long id)
    Returns an object that allows for the contents of an area (represented by the 'id' parameter) to be read and written.
    protected long
    getNextAreaHeader(long pointer, long[] header)
    Sets the 'header' array with information from the next header to the given pointer, and returns a pointer to the next area.
    protected long
    getPreviousAreaHeader(long pointer, long[] header)
    Sets the 'header' array with information from the previous header to the given pointer, and returns a pointer to the previous area.
    protected abstract void
    Internally closes the backing area.
    protected abstract void
    internalOpen(boolean read_only)
    Internally opens the backing area.
    protected static boolean
    Returns true if the given area size is valid.
    boolean
    Returns true if the store was closed cleanly.
    boolean
    Opens the data store.
    void
    Opens/scans the store looking for any errors with the layout.
    protected void
    Reads the bins from the header information in the file.
    protected abstract int
    readByteArrayFrom(long position, byte[] buf, int off, int len)
    Reads a byte array from the given position in the file.
    protected abstract int
    readByteFrom(long position)
    Reads a byte from the given position in the file.
    protected void
    reboundArea(long pointer, long[] header, boolean write_headers)
    Rebounds the given area with the given header information.
    protected abstract void
    setDataAreaSize(long length)
    Sets the size of the data area.
    protected void
    splitArea(long pointer, long new_boundary)
    Splits an area pointed to by 'pointer' at a new boundary point.
    void
    statsScan(HashMap properties)
    Performs an extensive lookup on all the tables in this store and sets a number of properties in the given HashMap (property name(String) -> property description(Object)).
    long
    Returns the total allocated space since the file was openned.
    protected void
    Updates all bins to the data area header area.
    protected void
    writeBinIndex(int index)
    Updates the given bin index to the data area header area.
    protected abstract void
    writeByteArrayTo(long position, byte[] buf, int off, int len)
    Writes a byte array to the given position in the file.
    protected abstract void
    writeByteTo(long position, int b)
    Writes a byte to the given position in the file.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface com.mckoi.store.Store

    lockForWrite, unlockForWrite
  • Field Details

    • free_bin_list

      protected long[] free_bin_list
      The free bin list contains 128 entries pointing to the first available block in the bin. If the list item contains -1 then there are no free blocks in the bin.
    • wilderness_pointer

      protected long wilderness_pointer
      A pointer to the wilderness area (the last deleted area in the store), or -1 if there is no wilderness area.
    • read_only

      protected boolean read_only
      True if this is read-only.
    • total_allocated_space

      protected long total_allocated_space
      The total amount of allocated space within this store since the store was openned. Note that this could be a negative amount if more space was freed than allocated.
    • DATA_AREA_OFFSET

      protected static final long DATA_AREA_OFFSET
      The offset into the file that the data areas start.
      See Also:
    • FIXED_AREA_OFFSET

      protected static final long FIXED_AREA_OFFSET
      The offset into the file of the 64 byte fixed area.
      See Also:
    • BIN_AREA_OFFSET

      protected static final long BIN_AREA_OFFSET
      The offset into the file that the bin area starts.
      See Also:
    • MAGIC

      protected static final int MAGIC
      The magic value.
      See Also:
    • header_buf

      protected final byte[] header_buf
    • BIN_ENTRIES

      protected static final int BIN_ENTRIES
  • Constructor Details

    • AbstractStore

      protected AbstractStore(boolean read_only)
      Constructs the store.
  • Method Details

    • open

      public boolean open() throws IOException
      Opens the data store. Returns true if the store did not close cleanly.
      Throws:
      IOException
    • close

      public void close() throws IOException
      Closes the store.
      Throws:
      IOException
    • isValidBoundarySize

      protected static boolean isValidBoundarySize(long size)
      Returns true if the given area size is valid. Currently the criteria for a valid boundary size is (size >= 24) and (size % 8 == 0) and (size invalid input: '<' 200 gigabytes)
    • openScanAndFix

      public void openScanAndFix(UserTerminal terminal) throws IOException
      Opens/scans the store looking for any errors with the layout. If a problem with the store is detected, it attempts to fix it.
      Throws:
      IOException
    • statsScan

      public void statsScan(HashMap properties) throws IOException
      Performs an extensive lookup on all the tables in this store and sets a number of properties in the given HashMap (property name(String) -> property description(Object)). This should be used for store diagnostics.

      Assume the store is open.

      Throws:
      IOException
    • getAllAreas

      public List getAllAreas() throws IOException
      Returns a List of Long objects that contain a complete list of all areas in the store. This is useful for checking if a given pointer is valid or not. The returned list is sorted from start area to end area.
      Specified by:
      getAllAreas in interface Store
      Throws:
      IOException
    • findAllocatedAreasNotIn

      public ArrayList findAllocatedAreasNotIn(ArrayList list) throws IOException
      Scans the area list, and any areas that aren't deleted and aren't found in the given ArrayList are returned as leaked areas. This is a useful method for finding any leaks in the store.
      Throws:
      IOException
    • totalAllocatedSinceStart

      public long totalAllocatedSinceStart()
      Returns the total allocated space since the file was openned.
    • internalOpen

      protected abstract void internalOpen(boolean read_only) throws IOException
      Internally opens the backing area. If 'read_only' is true then the store is openned in read only mode.
      Throws:
      IOException
    • internalClose

      protected abstract void internalClose() throws IOException
      Internally closes the backing area.
      Throws:
      IOException
    • readByteFrom

      protected abstract int readByteFrom(long position) throws IOException
      Reads a byte from the given position in the file.
      Throws:
      IOException
    • readByteArrayFrom

      protected abstract int readByteArrayFrom(long position, byte[] buf, int off, int len) throws IOException
      Reads a byte array from the given position in the file. Returns the number of bytes read.
      Throws:
      IOException
    • writeByteTo

      protected abstract void writeByteTo(long position, int b) throws IOException
      Writes a byte to the given position in the file.
      Throws:
      IOException
    • writeByteArrayTo

      protected abstract void writeByteArrayTo(long position, byte[] buf, int off, int len) throws IOException
      Writes a byte array to the given position in the file.
      Throws:
      IOException
    • endOfDataAreaPointer

      protected abstract long endOfDataAreaPointer() throws IOException
      Returns a pointer to the end of the current data area.
      Throws:
      IOException
    • setDataAreaSize

      protected abstract void setDataAreaSize(long length) throws IOException
      Sets the size of the data area.
      Throws:
      IOException
    • checkPointer

      protected void checkPointer(long pointer) throws IOException
      Checks the pointer is valid.
      Throws:
      IOException
    • readBins

      protected void readBins() throws IOException
      Reads the bins from the header information in the file.
      Throws:
      IOException
    • writeAllBins

      protected void writeAllBins() throws IOException
      Updates all bins to the data area header area.
      Throws:
      IOException
    • writeBinIndex

      protected void writeBinIndex(int index) throws IOException
      Updates the given bin index to the data area header area.
      Throws:
      IOException
    • getAreaHeader

      protected void getAreaHeader(long pointer, long[] header) throws IOException
      Sets the 'header' array with information from the header of the given pointer.
      Throws:
      IOException
    • getPreviousAreaHeader

      protected long getPreviousAreaHeader(long pointer, long[] header) throws IOException
      Sets the 'header' array with information from the previous header to the given pointer, and returns a pointer to the previous area.
      Throws:
      IOException
    • getNextAreaHeader

      protected long getNextAreaHeader(long pointer, long[] header) throws IOException
      Sets the 'header' array with information from the next header to the given pointer, and returns a pointer to the next area.
      Throws:
      IOException
    • reboundArea

      protected void reboundArea(long pointer, long[] header, boolean write_headers) throws IOException
      Rebounds the given area with the given header information. If 'write_headers' is true, the header (header[0]) is changed. Note that this shouldn't be used to change the size of a chunk.
      Throws:
      IOException
    • coalescArea

      protected void coalescArea(long pointer, long size) throws IOException
      Coalesc one or more areas into a larger area. This alters the boundary of the area to encompass the given size.
      Throws:
      IOException
    • expandDataArea

      protected long expandDataArea(long minimum_size) throws IOException
      Expands the data area by at least the minimum size given. Returns the actual size the data area was expanded by.
      Throws:
      IOException
    • splitArea

      protected void splitArea(long pointer, long new_boundary) throws IOException
      Splits an area pointed to by 'pointer' at a new boundary point.
      Throws:
      IOException
    • createArea

      public AreaWriter createArea(long size) throws IOException
      Description copied from interface: Store
      Allocates a block of memory in the store of the specified size and returns an AreaWriter object that can be used to initialize the contents of the area. Note that an area in the store is undefined until the 'finish' method is called in AreaWriter.
      Specified by:
      createArea in interface Store
      Parameters:
      size - the amount of memory to allocate.
      Returns:
      an AreaWriter object that allows the area to be setup.
      Throws:
      IOException - if not enough space available to create the area or the store is read-only.
    • deleteArea

      public void deleteArea(long id) throws IOException
      Description copied from interface: Store
      Deletes an area that was previously allocated by the 'createArea' method by the area id. Once an area is deleted the resources may be reclaimed. The behaviour of this method is undefined if the id doesn't represent a valid area.
      Specified by:
      deleteArea in interface Store
      Parameters:
      id - the identifier of the area to delete.
      Throws:
      IOException - (optional) if the id is invalid or the area can not otherwise by deleted.
    • getAreaInputStream

      public InputStream getAreaInputStream(long id) throws IOException
      Description copied from interface: Store
      Returns an InputStream implementation that allows for the area with the given identifier to be read sequentially. The behaviour of this method, and InputStream object, is undefined if the id doesn't represent a valid area.

      When 'id' is -1 then a fixed area (64 bytes in size) in the store is returned. The fixed area can be used to store important static static information.

      Specified by:
      getAreaInputStream in interface Store
      Parameters:
      id - the identifier of the area to read, or id = -1 is a 64 byte fixed area in the store.
      Returns:
      an InputStream that allows the area to be read from the start.
      Throws:
      IOException - (optional) if the id is invalid or the area can not otherwise be accessed.
    • getArea

      public Area getArea(long id) throws IOException
      Description copied from interface: Store
      Returns an object that allows for the contents of an area (represented by the 'id' parameter) to be read. The behaviour of this method, and Area object, is undefined if the id doesn't represent a valid area.

      When 'id' is -1 then a fixed area (64 bytes in size) in the store is returned. The fixed area can be used to store important static static information.

      Specified by:
      getArea in interface Store
      Parameters:
      id - the identifier of the area to read, or id = -1 is a 64 byte fixed area in the store.
      Returns:
      an Area object that allows access to the part of the store.
      Throws:
      IOException - (optional) if the id is invalid or the area can not otherwise be accessed.
    • getMutableArea

      public MutableArea getMutableArea(long id) throws IOException
      Description copied from interface: Store
      Returns an object that allows for the contents of an area (represented by the 'id' parameter) to be read and written. The behaviour of this method, and MutableArea object, is undefined if the id doesn't represent a valid area.

      When 'id' is -1 then a fixed area (64 bytes in size) in the store is returned. The fixed area can be used to store important static static information.

      Specified by:
      getMutableArea in interface Store
      Parameters:
      id - the identifier of the area to access, or id = -1 is a 64 byte fixed area in the store.
      Returns:
      a MutableArea object that allows access to the part of the store.
      Throws:
      IOException - (optional) if the id is invalid or the area can not otherwise be accessed.
    • lastCloseClean

      public boolean lastCloseClean()
      Description copied from interface: Store
      Returns true if the store was closed cleanly. This is important information that may need to be considered when reading information from the store. This is typically used to issue a scan on the data in the store when it is not closed cleanly.
      Specified by:
      lastCloseClean in interface Store