storage.pl -- Store files on behalve of web clients

The file store needs to deal with versioning and meta-data. This is achieved using gitty.pl, a git-like content-base store that lacks git's notion of a tree. I.e., all files are considered individual and have their own version.

 web_storage(+Request) is det[private]

Restfull HTTP handler to store data on behalf of the client in a hard-to-guess location. Returns a JSON object that provides the URL for the data and the plain file name. Understands the HTTP methods GET, POST, PUT and DELETE.

 update_error(+Error, +Storage, +Data, +File, +URL)[private]

If error signals an edit conflict, prepare an HTTP 409 Conflict page

 follow(+Commit, +SaveDict) is det[private]

Broadcast follow(DocID, ProfileID, [update,chat]) if the user wishes to follow the file associated with Commit.

 request_file(+Request, +GittyDir, -File) is det[private]

Extract the gitty file referenced from the HTTP Request.

Errors

- HTTP 404 exception

 meta_data(+Dict, -Meta, +Options) is det[private]

 meta_data(+Store, +Dict, -PrevMeta, -Meta, +Options) is det[private]

Gather meta-data from the Request (user, peer, identity) and provided meta-data. Illegal and unknown values are ignored.

The meta_data/5 version is used to add information about a fork.

Arguments:

Dict	- represents the JSON document posted and contains the content (data) and meta data (meta).

 storage_get(+Request, +Format, +Options) is det[private]

HTTP handler that returns information a given gitty file.

Arguments:

Format

- is one of swish Serve file embedded in a SWISH application raw Serve the raw file json Return a JSON object with the keys data and meta history(Depth, IncludeHASH) Return a JSON description with the change log diff(RelTo) Reply with diff relative to RelTo. Default is the previous commit.

 gitty_data_or_default(+Dir, +Type, +FileOrHash, -Code, -Meta)[private]

Read a file from the gitty store. I the file is not present, a default may be provided gitty/File in the config directory.

 chat_count(+Meta, -ChatCount) is det[private]

True when ChatCount is the number of chat messages available about Meta.

 random_filename(-Name) is det[private]

Return a random file name from plain nice ASCII characters.

 swish_show(+Options, +Request)[private]

Hande a document. First calls the hook open_hook/2 to rewrite the document. This is used for e.g., permahashes.

 storage_file(?File) is nondet

 storage_file_extension(?File, ?Extension) is nondet

 storage_file_extension_head(?File, ?Extension, -Head) is nondet

 storage_file(+File, -Data, -Meta) is semidet

 storage_meta_data(+File, -Meta) is semidet

True if File is known in the store.

Arguments:

Data	- is a string holding the content of the file
Meta	- is a dict holding the meta data about the file.

 storage_commit(+Hash, -Meta) is semidet

Load the commit data for Hash. This version does not tell us whether Hash is the HEAD or not.

 storage_meta_property(+Meta, -Property)

True when Meta has Property. Defined properties are:

peer(Atom)

Peer address that last saved the file -

 storage_store_term(+Term, -Hash) is det

 storage_load_term(+Hash, -Term) is det

Add/retrieve terms from the gitty store. This is used to create permanent links to arbitrary objects.

 use_gitty_file(+File) is det

 use_gitty_file(+File, +Options) is det

Load a file from the Gitty store. Options are passed to load_files/2. Additional options are:

watch(+Boolean)

If true (default), reload the file if the user saves it.

 collect_messages_as_json(+Goal, -Messages)[private]

Run Goal, collecting messages as produced by print_message/2 in Messages as JSON terms.

 storage_fsck

Enumerate and check the consistency of the entire store.

 storage_repack is det

 storage_repack(+Options) is det

Repack the storage directory. Currently only supports the files driver. For database drivers this is supposed to be handled by the database.

 storage_unpack

Unpack all packed objects of the store. Currently only supports the files driver. For database drivers this is supposed to be handled by the database.

 swish_search:typeahead(+Set, +Query, -Match, +Options) is nondet[multifile]

Find files using typeahead from the SWISH search box. This version defines the following sets:

• file: Search the store for matching file names, matching tag or title.
• store_content: Search the content of the store for matching lines.

To be done

- caching?

- We should only demand public on public servers.

 source_list(+Request)[private]

List source files. Request parameters:

q(Query)

Query is a string for which the following sub strings are treated special:

"..."

A quoted string is taken as a string search $ /.../[xim]* Regular expression search

tag:Tag

Must have tag containing

type:Type

Limit to one of pl, swinb or lnk

user:User

Must have user containing. If User is me must be owned by current user

name:Name

Must have name containing

o(Order)

Order by time (default), name, author or type

offset(+Offset)

limit(+Limit)

display_name

avatar

Weak identity parameters used to identify own documents that are also weakly identified.

Reply is a JSON object containing count (total matches), cpu (CPU time) and matches (list of matching sources)

To be done

- Search the content when searching a .lnk file?

- Speedup expensive searches. Cache? Use external DB?

 visible(+FileMeta, +Auth, +MetaConstraints) is semidet[private]

 owns(+Auth, +Meta, ?How) is semidet[private]

True if the file represented by Meta is owned by the user identified as Auth. If this is a strong identity we must give a strong answer.

To be done

- Weaker identity on the basis of author, avatar properties and/or IP properties.

 matches_meta(+Source, +Auth, +Query) is semidet[private]

True when Source matches the meta-data requirements

 type_constraint(+Query0, -Query, -Type) is det[private]

Extract the type constraints from the query as we can handle that efficiently.

 parse_query(+String, -Query) is det[private]

Parse a query, resulting in a list of Name(Value) pairs. Name is one of tag, user, type, string or regex.

To be done

- : Should we allow for logical combinations?

 source_modified(+Request)[private]

Reply with the last modification time of the source repo. If there is no modification we use the time the server was started.

This is a poor men's solution to keep the client cache consistent. Need to think about a better way to cache searches client and/or server side.

Re-exported predicates

The following predicates are exported from this file while their implementation is defined in imported modules or non-module files loaded by this module.

 storage_file(?File) is nondet

 storage_file_extension(?File, ?Extension) is nondet

 storage_file_extension_head(?File, ?Extension, -Head) is nondet

 storage_file(+File, -Data, -Meta) is semidet

 storage_meta_data(+File, -Meta) is semidet

True if File is known in the store.

Arguments:

Data	- is a string holding the content of the file
Meta	- is a dict holding the meta data about the file.

 storage_file(?File) is nondet

 storage_file_extension(?File, ?Extension) is nondet

 storage_file_extension_head(?File, ?Extension, -Head) is nondet

 storage_file(+File, -Data, -Meta) is semidet

 storage_meta_data(+File, -Meta) is semidet

True if File is known in the store.

Arguments:

Data	- is a string holding the content of the file
Meta	- is a dict holding the meta data about the file.

 storage_file(?File) is nondet

 storage_file_extension(?File, ?Extension) is nondet

 storage_file_extension_head(?File, ?Extension, -Head) is nondet

 storage_file(+File, -Data, -Meta) is semidet

 storage_meta_data(+File, -Meta) is semidet

True if File is known in the store.

Arguments:

Data	- is a string holding the content of the file
Meta	- is a dict holding the meta data about the file.

 storage_file(?File) is nondet

 storage_file_extension(?File, ?Extension) is nondet

 storage_file_extension_head(?File, ?Extension, -Head) is nondet

 storage_file(+File, -Data, -Meta) is semidet

 storage_meta_data(+File, -Meta) is semidet

True if File is known in the store.

Arguments:

Data	- is a string holding the content of the file
Meta	- is a dict holding the meta data about the file.

 storage_store_term(+Term, -Hash) is det

 storage_load_term(+Hash, -Term) is det

Add/retrieve terms from the gitty store. This is used to create permanent links to arbitrary objects.

 use_gitty_file(+File) is det

 use_gitty_file(+File, +Options) is det

Load a file from the Gitty store. Options are passed to load_files/2. Additional options are:

watch(+Boolean)

If true (default), reload the file if the user saves it.

 storage_repack is det

 storage_repack(+Options) is det

Repack the storage directory. Currently only supports the files driver. For database drivers this is supposed to be handled by the database.