SDT-base         Contents     Functions              PDF Index

fjlock

• fjlock
• .delete[,.close]
• .file
• .lock(flag)
• .lockWait(fname,flag,icmax,dt)
• .locked
• .setFile
• .setUseNIO(flag)
• .tmpFile

Purpose

File lock handling object.

Syntax

ob=fjlock(fname); % initialize object
ob.lock(flag); % lock/unlock with flag
state=ob.locked; % lock status

Description

To avoid simultaneous file access or to help with keeping track of currently processed files, one can use fjlock to test file accessibility or to lock file accessibility to other processes.

fjlock behavior follows the following semantics

The lock is handled within the object, ensuring exclusivity even if several fjlock objects referring to the same file exist in different processes. The lock holder is thus a unique object independently from MATLAB sessions or processes. This leads to three distinct lock statuses:

• 0 file unlocked: no external lock found, and not locked in object. The file is accessible but should be locked to safely proceed.
• 1 file externally locked: no access possible, impossible to change the lock status until the lock holder has released it, the process testing accessibility should not proceed.
• 2 file lock within the object: the file is locked but this object is the lock holder, the process using this object may proceed safely.

When a lock holder fjlock is destroyed, the lock is released.

fjlock inherits the handle class, so that any copy refers to the same object (thus same lock status, holder, or destroyed). It is also recommended to use delete instead of clear to destroy the object. The clear command may postpone destruction and thus lock release in recent MATLAB versions.

File locking is never absolutely perfect, as OSes do not use transactional file systems. Besides, POSIX semantics are nowadays weakly enforced to optimize latency, especially over network access, and efficient strategies will depend on the OS. Several lock strategies are thus implemented with their own pros and cons.

• Java LockFile (flag=1) This implementation locks the file itself instead of generating a companion lock file. This is a very robust and attractive method but its effect is restrained within a specific JRE instance (one machine). The lock validity will thus be limited to all MATLAB instances on a specific computer. Once locked the file may become inaccessible to read to any process (even the one hodling the lock). This behavior has been observed on Windows10. A non recoverable segFault may also fail to release the file, that would then remain locked at the JRE level without release access. The JRE would then need to be restarted to recover accessibility.
• external lock file with Java IO (flag=2,useNIO=0) This implementation generates a lock file companion whose existence will define the lock status. To be really safe the lock file generation has to be atomic, here through the File.createNewFile() method. Specific care is taken to avoid issues related to weak atomicity to the limits of the file system. The lock is then valid with no network limit and cross-platform access. As the file itself is not locked no access issue will exist. This also means that nothing will prevent another rude process to access the file, or delete it, or delete the lock file. The success of this method is thus linked to the robustness of access test. File access recovery in case of object loss is here easily done by deleting the lock file companion.
• external lock file with Java NIO (flag=2,useNIO=1) This implementation is a variant to the IO implementation. The main characteristics are the same, but atomicity if realized using the NIO class. The Files.copy method is used instead of the Files.move method. In recent file system the latter method may be atomic but fails to throw exceptions if the target already exist thus failing the lock scheme. The Files.copy associated to an empty file works better, but no atomicity is guaranteed so that a risk of error or lock file corruption may exist, although very small. This method (new from Java7) is eventually limited to recent MATLAB versions. For Unix environments starting from MATLAB 8.2 (R2013b), for Windows environments from MATLAB 9.1 (R2016b).

To be robust to the possibility of several lock strategies used at once, strategy 2 overrides strategy 1 in the lock holder only, and the lock status is independent from the strategy employed. It is then impossible to lock a file if any lock is detected. One can switch in the lock holder from strategy 1 to 2 but not the other way around until the lock is released. Although very improbable, the lock hold could be lost during the switch. It is anyways recommended not to mix strategies within a given distributed procedure.

The default behavior assumes that to be locked, a file must exist. If a file gets deleted, any referred hold lock will be released. It is however sometimes interesting to place a lock on a non-existing file to protect its creation. This specific behavior only works with the external lock files strategies. The operation must then be explicitly called using flag=3. In such case the external lock file strategy is forced and a lock can be hold on a non-existing file.

fjlock

fjlock constructor. Calling fjlock will create a new fjlock object. One can provide a file name string fname, and a lock flag integrer on-the-fly.

ob1=fjlock; % create empty object
ob1=fjlock(fname); % refer to file fname, access tested
ob1=fjlock(fname,flag); % refer to file fname and try to lock with flag

.delete[,.close]

fjlock destruction (and callback). Releases the lock if the object is a lock holder prior to destruction. If no process refers to the object or when exiting MATLAB this method will be called too.

.file

Provide a reference file name. It is possible to change the file reference in an existing object, in such case, the hold locks will be released.

ob1.file=fname2; % change fname

.lock(flag)

Try to assign a lock flag to the referred file, and outputs the lock status. If the flag is set to false and hold lock is released, otherwise flag defines the lock strategy (and not the lock status per se).
status=obj.lock(flag).

If the file is locked externally (status set to 1), nothing will be performed, one can however try to lock in a wait loop until the lock hold (status 2) is obtained.

Depending on the initial lock status, flag setting will have the following effect

• file exists, initial status unlocked (status 0)
  • flag=0 nothing is done, file remains unlocked.
  • flag=1 lock with FileLock strategy, lock is hold with strategy 1.
  • flag=2 lock with external lock file, lock is hold with strategy 2.
  • flag=3 same as flag=2.
• file exists, initial status is externally locked (status 1)
  • flag=0 nothing is done, file remains externally locked.
  • flag=1 nothing is done, file remains externally locked.
  • flag=2 nothing is done, file remains externally locked.
  • flag=3 nothing is done, file remains externally locked.
• file exists, initial status is lock hold with FileLock strategy (status 2) (stra1)
  • flag=0 lock is released, file becomes unlocked.
  • flag=1 nothing is done, lock is hold.
  • flag=2 switch to external file lock strategy, java FileLock is released, lock is hold.
  • flag=3 same as flag=2.
• file exists, initial status is lock hold with external lock file strategy (status 2) (stra2)
  • flag=0 lock is released, file becomes unlocked.
  • flag=1 nothing is done (no strategy change), lock is hold.
  • flag=2 nothing is done, lock is hold.
  • flag=3 same as flag=2.
• file does not exist, initial status unlocked (status 0)
  • flag=0 nothing is done, file remains unlocked.
  • flag=1 nothing is done, file remains unlocked.
  • flag=2 nothing is done, file remains unlocked.
  • flag=3 lock with external lock file, lock is hold with strategy 2.
• file does not exist, initial status is lock hold with external lock file strategy (status 2) (stra2)
  • flag=0 lock is released, file becomes unlocked.
  • flag=1 nothing is done (no strategy change), lock is hold.
  • flag=2 nothing is done, lock is hold.
  • flag=3 nothing is done, lock is hold.

.lockWait(fname,flag,icmax,dt)

This functionality implements a file lock strategy with a wait loop for robustness. In practice, the user can call this command to implement a one-liner lock procedure that will hold until lock is successful or fail after a timeout. Syntax is [ob,id]=fjlock.lockWait(fname,flag,icmax,dt) with

• fname the file name to be locked, flag the lock flag. If this parameter is omitted, the flag defined by the environment variable SD_LOCK_MODE will be used, or if not defined, flag 2.
• icmax the maximum loop iterations to be performed. If omitted a maximum number of 1000 is used.
• dt the delay between two iterations. If omitted a delay of 0.1 second is used.

The total acceptable delay is thus given by icmax*dt.

Output ob is the fjlock object that will be needed to call for a release with ob.lock(0). Output id is the success flag, possible values are

• id=2 file successfully locked
• id=-1 file could not be locked, as the MATLAB session runs in -nojvm mode. File lock indeed requires java methods to run.
• id=-2 file could not be locked after timeout, as it is already locked by another session/thread.
• id=-3 file could not be locked due to an internal fjlock error. In such case, please report to SDTools if the problem persists.

.locked

Dynamically provides the object lock status associated to the referred file. Every call to .locked thus tests again file accessibility. The output is then the lock status.
status=obj.locked;

.setFile

Change the referred file in the object. If the object is a lock holder, the lock is released. No lock is performed on the newly referred file.
obj.file=fname;

.setUseNIO(flag)

Change the external file lock implementation strategy. If flag is false, the IO strategy will be used, the NIO otherwise. It is recommended to stick with the IO strategy.

.tmpFile

Generate a temporary file using java IO or NIO method. This method is used internally but can also be called externally to generate an empty temporary file with the methods available in Java. This is a variant to nas2up('tempname'), the difference being that .tmpFile directly creates an empty file.

f1=char(tmpFile(fjlock,sdtdef('tempdir'),'.mat')); % in tempdir with suffix .mat
f1=char(tmpFile(fjlock,sdtdef('tempdir'))); % in tempdir no suffix
f1=char(tmpFile(fjlock)); % in pwd, no suffix

Examples

% fjlock calls example
% Generate a file for illustration
f1=char(tmpFile(fjlock,sdtdef('tempdir'),'.mat'));
% Initialize object
ob=fjlock(f1) % dislays .file and .locked
status=ob.locked % 0:  unlocked
% lock the file
status=ob.lock(1) % status is 2
exist([f1 '.fjlock'],'file') % no external file
status=ob.lock(2) % status remains2
exist([f1 '.fjlock'],'file') % external file exists
% unlock the file
ob.lock(0) % status is 0
exist([f1 '.fjlock'],'file') % external file has been removed

% Now try with two objects
ob1=fjlock(f1) % new lock object
status2=ob1.lock(2) % status2 is 2 ob1 is lock holder
status=ob.locked % status passed to 1 file locked but not by ob
status=ob.lock(0) % status sill to 1 does nothing as ob is not holder
delete(ob1) % lock release when destructed
status=ob.locked % status passed to 0 as no lock exists anymore

f1=nas2up('tempname.mat');
ob=fjlock(f1,2);
status=ob.locked % 0: file does not exist
ob1=fjlock(f1,3);
status1=ob1.locked % 2: lock hold on non existing file
status=ob.locked % 1: external lock found
ob1.lock(0) % 0: lock released