sf::FileMappedStorage Class Reference

This class provides storing of huge amounts of data using file mapping.
Mapping only what is needed into memory. More...

#include <FileMappedStorage.h>

Classes
class	Lock
	Provides easy way to lock a pointer to the memory. More...

Public Types
typedef uint64_t	size_type
	Type used for sizes of storage.

Public Member Functions
	FileMappedStorage (size_type seg_sz, size_type blk_sz, size_type recycle=0)
	FileMappedStorage (const FileMappedStorage &)
	Copy constructor.
virtual	~FileMappedStorage ()
	Virtual destructor.
bool	reserve (size_type block_count)
	Reserves an amount of blocks and rounds it up to the nearest segment size.
void	flush ()
	Flushes all data stored by this instance.
bool	blockWrite (size_type ofs, size_type sz, const void *src)
	Function that store blocks in several segments when needed.
bool	blockRead (size_type ofs, size_type sz, void *dst) const
	Function that store blocks in several segments when needed.
size_type	getBlockCount () const
	Gets the amount of blocks reserved.
size_type	getSegmentCount () const
	Gets the amount of segments used by this instance.
size_type	getSegmentLocks () const
	Gets the accumulation of locks on segments.
size_type	getBlockSize () const
	Gets the block size in bytes of this instance.
size_type	getSegmentSize () const
	Gets the size of the segments in blocks of this instance.
size_type	getSize () const
FileMappedStorage::size_type	getRecycleCount () const
	Gets the amount of segments being recycled.
bool	setRecycleCount (size_type count)
	Sets the recycle segment count.
std::ostream &	writeStatus (std::ostream &os) const
	Writes the status to the output stream.

Static Public Attributes
static constexpr size_t	npos = std::numeric_limits<size_type>::max()
	Unsigned size value indicating not found or no index.

Detailed Description

This class provides storing of huge amounts of data using file mapping.
Mapping only what is needed into memory.

Typically used for data acquisition using digitisers with a vast stream of data in multiple channels.
In some cases the stored data must be kept together in blocks.
A block is a fixed length array of a certain type (mostly integers).
Blocks need to be atomic to for processing later.
All functions are block size oriented and multithreading safe and allowing data be processed in other threads.

The embedded sf::FileMappedStorage::Lock class provides a segment locking mechanism.

The example code below shows it simple usage.

// Where `_store` is `FileMappedStorage` data member.
bool MyProcess::process(size_type segment)
{
  // Create and try locking.
  Lock lock(_store, segment);
  // Boolean operator calling isAcquired() under the hood.
  if (lock)
  {
    // Get a typed pointer.
    auto p = lock.ptr<uint32_t>();
    //
    // Do the processing here.
    //
    // Signal processing success.
    return true;
  }
  // Signal segment was not acquired to be processed. Try again later.
  return false;
}

File mapping is done using a sf::IFileMapper derived class has implementations per OS and also a Qt version.

See also

sf::FileMappedStorage::Lock

sf::IFileMapper

sf::lnx::FileMapper

sf::win::FileMapper

sf::qt::FileMapper

Member Typedef Documentation

size_type

typedef uint64_t sf::FileMappedStorage::size_type

Type used for sizes of storage.

Constructor & Destructor Documentation

FileMappedStorage() [1/2]

sf::FileMappedStorage::FileMappedStorage	(	size_type	seg_sz,
		size_type	blk_sz,
		size_type	recycle = 0
	)

Initializing constructor.

Parameters

seg_sz	Sets the segment size of storage in blocks.
blk_sz	Is in bytes.
recycle	Is the maximum amount of segments allowed to be used for storing.

FileMappedStorage() [2/2]

sf::FileMappedStorage::FileMappedStorage

(

const FileMappedStorage &

)

Copy constructor.

~FileMappedStorage()

virtual sf::FileMappedStorage::~FileMappedStorage ( )

virtual

Virtual destructor.

Member Function Documentation

blockRead()

bool sf::FileMappedStorage::blockRead ( FileMappedStorage::size_type  ofs, FileMappedStorage::size_type  sz, void *  dst  ) const

inline

Function that store blocks in several segments when needed.

Parameters

ofs	offset in blocks
sz	Size in blocks
dst	Pointer to destination of data.

Returns

True on success.

blockWrite()

bool sf::FileMappedStorage::blockWrite ( FileMappedStorage::size_type  ofs, FileMappedStorage::size_type  sz, const void *  src  )

inline

Function that store blocks in several segments when needed.

Parameters

ofs	offset in blocks
sz	Size in blocks
src	Pointer to source of data.

Returns

True on success.

flush()

void sf::FileMappedStorage::flush

(

)

Flushes all data stored by this instance.

After this all segments are no longer and reserve() must be called again.

getBlockCount()

size_type sf::FileMappedStorage::getBlockCount

(

)

const

Gets the amount of blocks reserved.

Returns

Amount blocks.

See also

reserve

getBlockSize()

size_type sf::FileMappedStorage::getBlockSize

(

)

const

Gets the block size in bytes of this instance.

getRecycleCount()

FileMappedStorage::size_type sf::FileMappedStorage::getRecycleCount ( ) const

inline

Gets the amount of segments being recycled.

getSegmentCount()

size_type sf::FileMappedStorage::getSegmentCount

(

)

const

Gets the amount of segments used by this instance.

getSegmentLocks()

size_type sf::FileMappedStorage::getSegmentLocks

(

)

const

Gets the accumulation of locks on segments.

getSegmentSize()

size_type sf::FileMappedStorage::getSegmentSize

(

)

const

Gets the size of the segments in blocks of this instance.

getSize()

size_type sf::FileMappedStorage::getSize

(

)

const

Gets the total amount of bytes handled by this instance. Taking recycling into account.

Returns

Bytes

reserve()

bool sf::FileMappedStorage::reserve

(

size_type

block_count

)

Reserves an amount of blocks and rounds it up to the nearest segment size.

Shrinking is not possible with this method.

Parameters

block_count

Amount of blocks to reserve.

Returns

True if it succeeded to fulfill the request.

setRecycleCount()

bool sf::FileMappedStorage::setRecycleCount

(

size_type

count

)

Sets the recycle segment count.

Parameters

count

Amount of segments to recycle.

Returns

True on success.

writeStatus()

std::ostream & sf::FileMappedStorage::writeStatus

(

std::ostream &

os

)

const

Writes the status to the output stream.

Member Data Documentation

npos

constexpr size_t sf::FileMappedStorage::npos = std::numeric_limits<size_type>::max()

staticconstexpr

Unsigned size value indicating not found or no index.

---

The documentation for this class was generated from the following file:

• FileMappedStorage.h