A VFS_datastore subclass for storing data on a locally attached hard drive or NFS share. More...

#include <VFS_hd.h>

Inheritance diagram for VFS_HD:

Public Slots
virtual void	applyDiff (VFS_request *r)
	Apply a diff to a file. More...

Public Slots inherited from VFS_node
virtual void	applyDiff (VFS_request *r)
	Apply a diff received via subscription. More...

virtual void	executeRequest (VFS_request *t)
	Based on the VFS_request::requestType, execute the function associated with an operation. More...

void	notifySubscribers (VFS_node origin, VFS_request t)
	Propagate a diff to subscribers. More...

virtual void	receiveResponse (VFS_request *t)
	Once a VFS_request has been completed, a response will be issued back to its _origin. More...

void	remove (bool andDelete)
	Remove a child node. More...

virtual void	remove (VFS_node node=nullptr, QString name=nullptr, QString user="server")
	Remove a child node from this node. More...

virtual void	subtreeRequest (VFS_request *t)
	find() the target of a VFS_request, and execute the request More...

virtual void	unsubscribeAll (VFS_node *n)
	Remove all references to a subscriber from this node. More...

Public Member Functions
Q_INVOKABLE	VFS_HD (QString p, quint64 size=VFS_datastore_cache::DEFAULT_DATACACHE_SIZE, bool create=false, bool debug=false, int flushInterval=5000, int expireInterval=60000, bool raw=false, bool watch=false)
	Create a VFS_HD node, with an optional create flag. More...

virtual	~VFS_HD ()

const QString	path ()
	Return the _path of this node. More...

virtual QString	reportDetails ()
	Report the current cache usage. More...

Public Member Functions inherited from VFS_datastore
	VFS_datastore (quint64 size, bool debug=false)

virtual	~VFS_datastore ()

virtual VFS_node *	find (VFS_request *r)

virtual bool	isContainer ()
	All datastore subclasses are containers. More...

Public Member Functions inherited from VFS_node
Q_INVOKABLE	VFS_node ()
	The VFS_node constructor will add its instance to the VFS_node::__allNodes global node registry, observing thread safety rules. More...

virtual	~VFS_node ()
	VFS_node destructor. More...

virtual VFS_node *	append (QString name, VFS_node *node, bool containerCheck=true, QString user="server")
	Append a VFS_node as a child of this node. More...

QString	className ()
	Return the class name of a node. More...

virtual VFS_request *	createRequest (VFS_request::requestType type, QString path, QString user="unknown", QJsonDocument data=QJsonDocument(), QJsonObject metadata=QJsonObject(), bool dontDelete=false)
	Create a new VFS_request with this object as _origin. More...

VFS_node *	find (QString path)
	Find a node by string path. More...

VFS_node *	findChildWithName (QString name)
	Check if a child with a given name exists. More...

virtual VFS_node *	mount ()
	Mount this node. More...

QString	uniqueChildName (QString name)
	Generate a unique child name. More...

virtual VFS_node *	unmount ()
	Unmount this node. More...

virtual bool	validChildName (QString name)
	Check if a node name is valid. More...

Protected Member Functions
virtual void	fetchRange (VFS_request *r)
	Fetch a byte range within a file. More...

virtual QByteArray	icon ()
	The "disk" icon found in the VFS_icons library. More...

virtual void	ls (VFS_request *r)
	List the contents of a filesystem directory. More...

virtual void	metadata (VFS_request *r)
	Fetch the metadata for a file or directory. More...

virtual void	read (VFS_request *r)
	Read the contents of a directory or file. More...

virtual void	rm (VFS_request *r)
	Attempt to delete a file. More...

virtual void	submit (VFS_request *r)
	Submit data to a file, applying the data as a diff. More...

virtual void	subscribe (VFS_request *r)
	Subscribe to changes on a file or directory. More...

virtual void	write (VFS_request *r)
	Write data to a file. More...

Protected Member Functions inherited from VFS_node
virtual void	aclDefaults (VFS_request *r)
	Return default values and features associated wth this node. More...

void	addACLDefault (QJsonObject &acl, bool value, QString description="")
	Add a default value to the acl object. More...

void	addACLFeature (QJsonObject &acl, QString feature, bool value, QString description="")
	Add a feature to the acl object. More...

void	addACLFeatureGroup (QJsonObject &acl, QString feature, QString group, bool value)
	Add a feature group to the acl object. More...

void	addACLFeatureUser (QJsonObject &acl, QString feature, QString user, bool value)
	Add a feature user to the acl object. More...

void	addACLGroup (QJsonObject &acl, QString group, bool value)
	Add a group to the acl object. More...

void	addACLUser (QJsonObject &acl, QString user, bool value)
	Add a user to the acl object. More...

virtual void	issueRequest (VFS_node target, VFS_request t)
	Issue a VFS_request to its target. More...

virtual void	issueRequest (VFS_request *t)
	A convenience function. More...

virtual void	issueResponse (VFS_request *t)
	Once a request has been completed, issue a response. More...

virtual void	releaseLock (VFS_request *r)
	Release a lock on this node. More...

virtual void	report (VFS_request *r)
	Report debugging information about the current state of this node. More...

virtual void	requestLock (VFS_request *r)
	Request a lock on this node. More...

virtual void	unsubscribe (VFS_request *r)
	Remove an entry from this node's _subscription list. More...

virtual void	unsubscribePath (QString path)
	Unsubscribe all references to a subpath. More...

Protected Attributes
int	_expireInterval
	Interval for cache entries to expire, in milliseconds. More...

int	_flushInterval
	Interval for cache entries to flush, in milliseconds. More...

QString	_path
	The base prefix path for this VFS_HD node. More...

bool	_raw
	Whether to default to JSON mode or RAW mode, which can be overridden with metadata. More...

bool	_watch
	Whether to set the watch flag on cache entries. More...

Protected Attributes inherited from VFS_datastore
VFS_datastore_cache	_cache
	The data cache. More...

bool	_debug
	Debug mode. More...

Protected Attributes inherited from VFS_node
VFS_children	_children
	This node's children. More...

QMutex	_lock
	A recursive mutex that is local to this node. More...

VFS_subscriptionType	_subscribers
	This node's subscribers. These subscribers will receive diff notifications. More...

Static Protected Attributes
static QMimeDatabase	_mimeDatabase
	The mime database, used for metadata. More...

Additional Inherited Members
Signals inherited from VFS_node
void	diff (VFS_node origin, VFS_request t)
	Emit a diff, which will trigger notifySubscribers() for a mounted node. More...

void	finished (bool andDelete=false)
	Emitted if a thread fails to create its node, or a node is rm()'d, or any other reason a node has completed its lifecycle. It is deleted if `andDelete==true`. More...

void	mounted ()
	Emitted when a node is mount()ed. More...

void	unmounted (VFS_node *self)
	Emitted when a node is unmount()ed. More...

Static Public Member Functions inherited from VFS_datastore
static QString	applyDeltaDiff (QString data, QJsonObject delta, QString trace="", QString user="server")
	Apply a delta diff to a json object. More...

static QJsonObject	applyJsonDiff (QJsonObject obj, QJsonObject diff, QString trace="", QString user="server")
	Apply a json diff to a json object. More...

Static Public Member Functions inherited from VFS_node
static bool	__isNode (VFS_node *)
	Check to see if a node is in the global registry. More...

static QString	code (QString nodename, QString libname, QString &error)
	Fetch code or any other resource from a node. More...

Detailed Description

A VFS_datastore subclass for storing data on a locally attached hard drive or NFS share.

DOCME: General philosophy, do softlinks work, path math

Warning

FIXME: These rules need to be revised!

 ##Basic Rules and Concepts:

 For files:
 File Entries               | Action
 ------------------------|------------------------------
 ls() a file:               | return nothing
 create a file:             | *see below
 rm() a file:               | delete the file from the filesystem
 read() from file:  | return the file's contents
 write() to file:   | replace the file's contents
 metadata() a file: | fetch a file's metadata (which will basically be none unless file extensions become supported)
 submit() to file:  | apply a diff to a file's contents
 subscribe() to file:       | create a subscription entry
 unsubscribe() file:        | remove subscription entry

 For directories:
 Directory Entries  | Action
 ------------------------|-------------------------------
 ls() a dir:                | return an object of name=>isDir() pairs of children
 create a dir:              | *see below
 rm() a dir:                | delete the directory, but only if it's empty
 read() from dir:   | return directory listing as a json array, containing name, path, and container==bool... basically the same as an ls()
 write() to dir:            | *generate an error!
 metadata() a dir:  | fetch a directory's metadata (which will basically be none unless dir extensions become supported)
 submit() to dir:   | *generate an error, unless it's a metadata only (which will at some point be supported)
 subscribe() to dir:        | create a subscription.  New and removed directory entries will notify the subscriber
 unsubscribe() dir: | remove subscription entry

FIXME: NEEDS REVISING!

 ##Advanced concepts:

 Any directory that ends with a .rmn (ReMotoNode) or .rmd (ReMotoDir) extension has special magic.  This is the mechanism
 used to provide icons, metadata and permissions to otherwise normal files and directories that exist
 on a normal filesystem.  This method seems better than requiring special disk formatting with extended
 attributes or resource forks or hidden files.

 \code
 file.rmn or dir.rmd
    +- data                 json or binary or directory
    +- icon.ext             extension determines mime type
    +- metadata             json -- must contain type information, which will imply how diffs are applied
    +- permissions  json -- **needs to be implemented!
 \endcode

 Any entry with an .rmn/d extension will be listed without that extension, and its path will not include
 any component extensions.

 .rmn/d                     | Action
 ------------------------|-------------------------------
 ls() an rmn/d:             | return name of entry if rmn, or newline list of entries for rmd
 create an rmn/d:   | based on metadata, create an .rmn or file or .rmd directory.  Missing metadata will create non-rmn/d file
 rm() an rmn/d:             | if container, remove rmn structure entirely if it has no children.  Non container will be deleted entirely
 read() an rmn/d:   | return the contents of rmn/data or xml listing of rmd/data
 write() to rmn/d:  | replace contents of rmn/data or throw error if rmd/data
 metadata() an rmn/d:       | return the contents of rmn/metadata, which will also include rmn/icon as b64-encoded if it exists
 submit() to an rmn/d:      | apply diff to rmn/data using rules based on metadata type specs, or throw error if rmd/data
 subscribe() to rmn/d:      | subscribe to changes on data
 unsubscribe() rmn/d:       | unsubscribe to changes on data

Definition at line 52 of file VFS_hd.h.

Constructor & Destructor Documentation

◆ VFS_HD()

VFS_HD::VFS_HD	(	QString	p,
		quint64	size = `VFS_datastore_cache::DEFAULT_DATACACHE_SIZE`,
		bool	create = `false`,
		bool	debug = `false`,
		int	flushInterval = `5000`,
		int	expireInterval = `60000`,
		bool	raw = `false`,
		bool	watch = `false`
	)

explicit

Create a VFS_HD node, with an optional create flag.

Parameters

p	The base filesystem path
size	The cache size
create	Create flag
debug	Verbose debugging
flushInterval	Interval to flush to disk, in milliseconds.
expireInterval	Interval to expire the cache entry, in milliseconds.
raw	Force all operations on this node to be in raw mode, not JSON
watch	Watch the filesystem for changes to cached entries. This may have limitations depending on operating system.

If the create flag is set, an attempt will be made to create an empty directory to point this VFS_HD to.

Definition at line 441 of file VFS_hd.cpp.

◆ ~VFS_HD()

VFS_HD::~VFS_HD ( )

virtual

Definition at line 469 of file VFS_hd.cpp.

Member Function Documentation

◆ applyDiff

void VFS_HD::applyDiff ( VFS_request * r )

virtualslot

Apply a diff to a file.

Parameters

r	The VFS_request object

This calls submit() using the request object. This does not call VFS_node::applyDiff() because the submit function should emit a diff() as needed.

Note: There is something odd in this function, as it uses r->_originPath to do its path math.

Definition at line 1209 of file VFS_hd.cpp.

◆ fetchRange()

void VFS_HD::fetchRange ( VFS_request * r )

protectedvirtual

Fetch a byte range within a file.

Parameters

r	The VFS_request object

This method bypasses all caching.

It will not return chunks larger than VFS_HD_MAX_READABLE, which is 100mb.

r->_metadata["range"] should include an object with a chunk range:

//metadata to return 500 bytes of data:
{
    "range": {
        "s": 0,
        "e": 499
    }
}

Where s and e indicate start and end of the range, inclusive.

If s or e is missing, value of 0 will be used. If s or e is a negative value, the start will be an offset from the end of the file. If e is 0, the end of the file will be used. If s >= e, no data will be returned.

Definition at line 1252 of file VFS_hd.cpp.

◆ icon()

QByteArray VFS_HD::icon ( )

protectedvirtual

The "disk" icon found in the VFS_icons library.

Returns: The "disk" icon

See also: VFS_node::icon()

Reimplemented from VFS_node.

Definition at line 481 of file VFS_hd.cpp.

◆ ls()

void VFS_HD::ls ( VFS_request * r )

protectedvirtual

List the contents of a filesystem directory.

Parameters

r	The VFS_request object

The resulting list will be of the form:

{
   "entry1":true,
   "entry2":false,
   ... etc.
}

Where the true and false values correspond to whether an entry is a file or directory. The values are sorted alphabetically by the json serializer.

A file entry itself can't be used for ls(). This will generate an error and r->_success=false.

VFS_HD::ls() does not use the _cache. It will read the underlying filesystem and block during that time.

As a convenience for users browsing the filesystem (to select a project file, for instance), an ls() command may have r->_metadata["closest"]=true which will follow a search up a given r->_path to find the nearest existing folder to list. This is for the case where a file or folder has been deleted, but we don't want the user to be placed in the root folder of the VFS_HD when they browse again. If a candidate is found, r->_metadata["closestContainer"] is set to that path.

Optionally, if r->_metadata["sequence"]==true, the resulting list will be collapsed into a sequence view using rutils::sequenceListing().

Entries that the server user does not have permission to read are not included in the resulting list. Arguably they could be included with a different value, but downstream processes would need to be sensitive to that value, and the server can't read the contents of the file or its metadata anyway. In the case where some file was skipped due to permissions, r->_metadata["entriesExcluded"]=true is set. If r->_metadata["ignorePermissions"]=true, all entries will be included.

See also: VFS_node::ls()

Reimplemented from VFS_node.

Definition at line 548 of file VFS_hd.cpp.

◆ metadata()

void VFS_HD::metadata ( VFS_request * r )

protectedvirtual

Fetch the metadata for a file or directory.

Parameters

r	The VFS_request object

Calculate the path to a file by prefixing the r->_path fragment with the VFS_HD::_path value.

Uses VFS_node_type::getType() to determine a type for a path.

Note: A more comprehensive method will be developed in the future

Reimplemented from VFS_node.

Definition at line 896 of file VFS_hd.cpp.

◆ path()

const QString VFS_HD::path ( )

Return the _path of this node.

Returns: The _path of this node

Definition at line 507 of file VFS_hd.cpp.

◆ read()

void VFS_HD::read ( VFS_request * r )

protectedvirtual

Read the contents of a directory or file.

Parameters

r	The VFS_request object

Calculate the path to a file or directory by prefixing the r->_path fragment with the VFS_HD::_path value.

REVISE THIS FOR raw HD, as metadata["raw"] is deprecated

Attempt to fetch the cached contents of the file or directory, or create a new cache entry if none found. VFS_HD_cache_entry::getData() is used to return the data from the cache entry.

If r->_metadata["raw"]==true, the contents of the file will be placed in r->_data["data"].

If r->_metadata["createIfMissing"]==true, and the entry does not exist, the read() will be converted to a write() with a VFS_request::create type.

See also: VFS_node::read(); VFS_HD::write()

Todo:: Revise raw HD documentation

Reimplemented from VFS_node.

Definition at line 716 of file VFS_hd.cpp.

◆ reportDetails()

QString VFS_HD::reportDetails ( )

virtual

Report the current cache usage.

Returns: Return cache usage and cache entry count.

Reimplemented from VFS_datastore.

Definition at line 492 of file VFS_hd.cpp.

◆ rm()

void VFS_HD::rm ( VFS_request * r )

protectedvirtual

Attempt to delete a file.

Parameters

r	The VFS_request object

Calculate the path to a file by prefixing the r->_path fragment with the VFS_HD::_path value.

Attempt to remove (delete) the file from the filesystem, and set r->_success=true if it was possible. The parent directory has changed, so emit a diff() on the parent to notify any subscribers.

Warning: FIXME: Deleting a directory is undefined

An rm() request where r->_path=="", following all other VFS conventions, removes the VFS_HD node from the VFS.

In all cases, remove the cache entry for this path.

See also: VFS_node::rm()

Reimplemented from VFS_node.

Definition at line 1099 of file VFS_hd.cpp.

◆ submit()

void VFS_HD::submit ( VFS_request * r )

protectedvirtual

Submit data to a file, applying the data as a diff.

Parameters

r	The VFS_request object

Calculate the path to a file by prefixing the r->_path fragment with the VFS_HD::_path value.

Fetch or create a cache entry for this file, and call VFS_HD_cache_entry::getData(). Apply the r->_data as a diff using VFS_datastore::applyJsonDiff(), or if r->_metadata["method"]=="delta", use VFS_datastore::applyDeltaDiff().

A submit() to a directory is illegal.

If r->_success==true, emit a diff() of the applied data to subscribers.

See also: VFS_HD::write(); VFS_datastore::applyJsonDiff(); VFS_datastore::applyDeltaDiff()

Reimplemented from VFS_node.

Definition at line 994 of file VFS_hd.cpp.

◆ subscribe()

void VFS_HD::subscribe ( VFS_request * r )

protectedvirtual

Subscribe to changes on a file or directory.

Parameters

r	The VFS_request object

Calculate the path to a file by prefixing the r->_path fragment with the VFS_HD::_path value.

Create a QFileInfo on the path to determine if it exists, and if it does, use the base class VFS_node::subscribe() on that path.

If it does not exist and r->_metadata["createIfMissing"]==true, the request will be converted to a write() with a VFS_request::create command, and if successful, VFS_node::subscribe() is called.

See also: VFS_node::subscribe()

Reimplemented from VFS_node.

Definition at line 1168 of file VFS_hd.cpp.

◆ write()

void VFS_HD::write ( VFS_request * r )

protectedvirtual

Write data to a file.

Parameters

r	The VFS_request object

Calculate the path to a file by prefixing the r->_path fragment with the VFS_HD::_path value.

The contents of the file will be entirely replaced by r->_data as a serialized json string.

Data cannot be written to a directory, however a directory can be created if r->_requestType==VFS_request::create and r->_metadata["container"]==true.

Because this is a write() operation, a diff() is emitted.

Todo:: explain metadata options here

See also: VFS_HD::read()

Reimplemented from VFS_node.

Definition at line 809 of file VFS_hd.cpp.

Member Data Documentation

◆ _expireInterval

int VFS_HD::_expireInterval

protected

Interval for cache entries to expire, in milliseconds.

Definition at line 69 of file VFS_hd.h.

◆ _flushInterval

int VFS_HD::_flushInterval

protected

Interval for cache entries to flush, in milliseconds.

Definition at line 68 of file VFS_hd.h.

◆ _mimeDatabase

QMimeDatabase VFS_HD::_mimeDatabase

staticprotected

The mime database, used for metadata.

Definition at line 73 of file VFS_hd.h.

◆ _path

QString VFS_HD::_path

protected

The base prefix path for this VFS_HD node.

Definition at line 67 of file VFS_hd.h.

◆ _raw

bool VFS_HD::_raw

protected

Whether to default to JSON mode or RAW mode, which can be overridden with metadata.

Definition at line 70 of file VFS_hd.h.

◆ _watch

bool VFS_HD::_watch

protected

Whether to set the watch flag on cache entries.

Definition at line 71 of file VFS_hd.h.

The documentation for this class was generated from the following files:

Public Slots

Public Member Functions

Protected Member Functions

Protected Attributes

Static Protected Attributes

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ VFS_HD()

◆ ~VFS_HD()

Member Function Documentation

◆ applyDiff

◆ fetchRange()

◆ icon()

◆ ls()

◆ metadata()

◆ path()

◆ read()

◆ reportDetails()

◆ rm()

◆ submit()

◆ subscribe()

◆ write()

Member Data Documentation

◆ _expireInterval

◆ _flushInterval

◆ _mimeDatabase

◆ _path

◆ _raw

◆ _watch