sol.models.bio -- Batched I/O

This module implements some utilities, mainly related to importing and exporting tourneys data in a portable format.

Scarry used an INI file and we had several drawbacks with it, mainly because sending them with e-mail would result in data corruption.

SoL uses YAML or JSON instead, by default compressing the outcome with gzip.

sol.models.bio.save_changes(sasess, request, modified, deleted, clogger=<Logger sol.models.bio.changes (WARNING)>)

Save insertions, changes and deletions to the database.

Parameters

  • sasess -- the SQLAlchemy session

  • request -- the Pyramid web request

  • modified -- a sequence of record changes, each represented by a tuple of two items, the PK name and a dictionary with the modified fields; if the value of the PK field is null or 0 then the record is considered new and will be inserted instead of updated

  • deleted -- a sequence of deletions, each represented by a tuple of two items, the PK name and the ID of the record to be removed

  • clogger -- where to log applied changes

Return type

tuple

Returns

three lists, respectively inserted, modified and deleted record IDs, grouped in a dictionary keyed on PK name

sol.models.bio.backup(sasess, pdir, edir, location=None, keep_only_if_changed=True, only_played_tourneys=False, serialization_format='yaml', native_when_possible=False, keep_max=30)

Dump almost everything in a ZIP file.

Parameters

  • sasess -- a SQLAlchemy session

  • pdir -- the base path of the portrait images, sol.portraits_dir

  • edir -- the base path of the emblem images, sol.emblems_dir

  • location -- either None or a string

  • keep_only_if_changed -- a boolean flag

  • only_played_tourneys -- a boolean flag

  • serialization_format (str) -- either yaml or json

  • native_when_possible (bool) -- if True, use the native interface to perform the backup

  • keep_max (int) -- the number of previous backups that will be kept

Return type

bytes

Returns

the ZIP archive

This function builds a ZIP archive containing a database dump and two subdirectories portraits and emblems, respectively containing the images associated to the players and to the clubs. When native_when_possible is True and the underlying system permits it the dump is a complete backup of the database, otherwise it is a standard .sol dump made with dump_sol() named everything.sol with all tourneys and all players (that is, not just those who actually played).

If location is given, it may be either the full path name of the output file where the backup will be written or the path of a directory. In the latter case the file name will be automatically computed using current time, giving something like sol-backup_2014-02-03T14:35:12.zip.

When keep_only_if_changed is ``True` (the default) and location is a directory, the newly generated backup will be compared with the previous one (if there is at least one, of course) and if nothing has changed it will be removed.

When only_played_tourneys is True (the default is False), the tourneys "in preparation" (that is, those without played matches) are ignored and not included in the dump.

At the end, only the the most recent keep_max backups will be kept, deleting those in excess, starting from the oldest.

sol.models.bio.restore(sasess, pdir=None, edir=None, url=None, content=None, idowner=None)

Restore everything from a backup.

Parameters

  • sasess -- a SQLAlchemy session

  • pdir -- the base path of the portrait images, sol.portraits_dir

  • edir -- the base path of the emblem images, sol.emblems_dir

  • url -- the URL of the file containing the archive, or None

  • content -- the content of the archive

  • idowner -- the ID of the responsible for newly created instances

Return type

tuple

Returns

the list of loaded tourney instances and the number of skipped tourneys

This reads the ZIP created by backup() and loads its content into the database, writing the images in the right place (pre-existing images won't be overwritten, though).

sol.models.bio.load_sol(sasess, url=None, content=None, restore=False, idowner=None)

Load the archive exported from SoL.

Parameters

  • sasess -- a SQLAlchemy session

  • url -- the URL of the .sol (or .sol.gz) file

  • content -- the content of a .sol (or .sol.gz) file

  • restore -- whether this is a restore, False by default

  • idowner -- the ID of the responsible for newly created instances

Return type

tuple

Returns

the list of loaded tourney instances and number of skipped tournaments

If content is not specified, it will be loaded with urlopen() from the given url.

Normally only missing data is updated, except when restore is True.

sol.models.bio.dump_sol(tourneys, gzipped=False, serialization_format='yaml')

Dump tourneys as a YAML or JSON document.

Parameters

  • tourneys -- the sequence of tourneys to dump

  • gzipped -- a boolean indicating whether the output will be compressed with gzip

  • serialization_format -- a string, either yaml or json

Return type

bytes

Returns

the YAML/JSON document, possibly gzipped

class sol.models.bio.Serializer

Serialize some SoL entities as flat dictionaries.

addChampionship(championship)

Serialize a championship, if not already done.

Parameters

club -- a Championship instance

Return type

int

Returns

an integer marker that identify the given championship

addClub(club)

Serialize a club, if not already done.

Parameters

club -- a Club instance

Return type

int

Returns

an integer marker that identify the given club

addPlayer(player)

Serialize a player, if not already done.

Parameters

player -- a Player instance

Return type

int

Returns

an integer marker that identify the given player

addRate(rate)

Serialize a rate, if not already done.

Parameters

rate -- a Rate instance

Return type

int

Returns

an integer marker that identify the given rate

addRating(rating)

Serialize a rating, if not already done.

Parameters

rating -- a Rating instance

Return type

int

Returns

an integer marker that identify the given rating

addTourney(tourney)

Serialize a tourney, if not already done.

Parameters

tourney -- a Tourney instance

Return type

int

Returns

an integer marker that identify the given tourney

addUser(user)

Serialize a user, if not already done.

Parameters

user -- a User instance

Return type

int

Returns

an integer marker that identify the given user

championships

A list of serialized championships

clubs

A list of serialized clubs

id_map

An hash mapping a particular instance to its serial marker

modified

Most recent modification timestamp of any serialized entity

players

A list of serialized players

rates

A list of serialized rates

ratings

A list of serialized ratings

tourneys

A list of serialized tourneys

users

A list of serialized users

class sol.models.bio.Deserializer(session, idowner, update_only_missing_fields)

Deserialize a flat representation of some SoL entities.

addChampionship(schampionship)

Deserialize a Championship.

Parameters

schampionship -- a dictionary containing the flatified representation

Return type

Championship

Returns

either an existing or a new instance

addClub(sclub)

Deserialize a Club.

Parameters

sclub -- a dictionary containing the flatified representation

Return type

Club

Returns

either an existing or a new instance

addPlayer(splayer)

Deserialize a Player.

Parameters

splayer -- a dictionary containing the flatified representation

Return type

Player

Returns

either an existing or a new instance

addRate(srate)

Deserialize a Rate.

Parameters

srate -- a dictionary containing the flatified representation

Return type

Rate

Returns

either an existing or a new instance

addRating(srating)

Deserialize a Rating.

Parameters

srating -- a dictionary containing the flatified representation

Return type

Rating

Returns

either an existing or a new instance

addTourney(stourney)

Deserialize a Tourney.

Parameters

stourney -- a dictionary containing the flatified representation

Return type

Tourney

Returns

either an existing or a new instance

addUser(suser)

Deserialize a User.

Parameters

suser -- a dictionary containing the flatified representation

Return type

User

Returns

either an existing or a new instance

championships

A list of championships instances.

clubs

A list of clubs instances.

idowner

The ID of the owner of newly created instances.

players

A list of players instances.

rates

A list of rates instances.

ratings

A list of ratings instances.

session

The SQLAlchemy session.

skipped

The number of skipped tournaments, because already present.

tourneys

A list of tourneys instances.

update_only_missing_fields

A boolean flag, whether only missing fields will be updated.

users

A list of users instances.