|
33
|
1
|
|
|
2 :mod:`lockfile` --- Platform-independent file locking
|
|
|
3 =====================================================
|
|
|
4
|
|
|
5 .. module:: lockfile
|
|
|
6 :synopsis: Platform-independent file locking
|
|
|
7 .. moduleauthor:: Skip Montanaro <skip@pobox.com>
|
|
|
8 .. sectionauthor:: Skip Montanaro <skip@pobox.com>
|
|
|
9
|
|
|
10
|
|
|
11 .. note::
|
|
|
12
|
|
|
13 This package is pre-release software. Between versions 0.8 and 0.9 it
|
|
|
14 was changed from a module to a package. It is quite possible that the
|
|
|
15 API and implementation will change again in important ways as people test
|
|
|
16 it and provide feedback and bug fixes. In particular, if the mkdir-based
|
|
|
17 locking scheme is sufficient for both Windows and Unix platforms, the
|
|
|
18 link-based scheme may be deleted so that only a single locking scheme is
|
|
|
19 used, providing cross-platform lockfile cooperation.
|
|
|
20
|
|
|
21 .. note::
|
|
|
22
|
|
|
23 The implementation uses the `with` statement, both in the tests and in the
|
|
|
24 main code, so will only work out-of-the-box with Python 2.5 or later.
|
|
|
25 However, the use of the `with` statement is minimal, so if you apply the
|
|
|
26 patch in the included 2.4.diff file you can use it with Python 2.4. It's
|
|
|
27 possible that it will work in Python 2.3 with that patch applied as well,
|
|
|
28 though the doctest code relies on APIs new in 2.4, so will have to be
|
|
|
29 rewritten somewhat to allow testing on 2.3. As they say, patches welcome.
|
|
|
30 ``;-)``
|
|
|
31
|
|
|
32 The :mod:`lockfile` package exports a :class:`LockFile` class which provides
|
|
|
33 a simple API for locking files. Unlike the Windows :func:`msvcrt.locking`
|
|
|
34 function, the Unix :func:`fcntl.flock`, :func:`fcntl.lockf` and the
|
|
|
35 deprecated :mod:`posixfile` module, the API is identical across both Unix
|
|
|
36 (including Linux and Mac) and Windows platforms. The lock mechanism relies
|
|
|
37 on the atomic nature of the :func:`link` (on Unix) and :func:`mkdir` (On
|
|
|
38 Windows) system calls. It also contains several lock-method-specific
|
|
|
39 modules: :mod:`lockfile.linklockfile`, :mod:`lockfile.mkdirlockfile`, and
|
|
|
40 :mod:`lockfile.sqlitelockfile`, each one exporting a single class. For
|
|
|
41 backwards compatibility with versions before 0.9 the :class:`LinkFileLock`,
|
|
|
42 :class:`MkdirFileLock` and :class:`SQLiteFileLock` objects are exposed as
|
|
|
43 attributes of the top-level lockfile package, though this use was deprecated
|
|
|
44 starting with version 0.9 and will be removed in version 1.0.
|
|
|
45
|
|
|
46 .. note::
|
|
|
47
|
|
|
48 The current implementation uses :func:`os.link` on Unix, but since that
|
|
|
49 function is unavailable on Windows it uses :func:`os.mkdir` there. At
|
|
|
50 this point it's not clear that using the :func:`os.mkdir` method would be
|
|
|
51 insufficient on Unix systems. If it proves to be adequate on Unix then
|
|
|
52 the implementation could be simplified and truly cross-platform locking
|
|
|
53 would be possible.
|
|
|
54
|
|
|
55 .. note::
|
|
|
56
|
|
|
57 The current implementation doesn't provide for shared vs. exclusive
|
|
|
58 locks. It should be possible for multiple reader processes to hold the
|
|
|
59 lock at the same time.
|
|
|
60
|
|
|
61 The module defines the following exceptions:
|
|
|
62
|
|
|
63 .. exception:: Error
|
|
|
64
|
|
|
65 This is the base class for all exceptions raised by the :class:`LockFile`
|
|
|
66 class.
|
|
|
67
|
|
|
68 .. exception:: LockError
|
|
|
69
|
|
|
70 This is the base class for all exceptions raised when attempting to lock
|
|
|
71 a file.
|
|
|
72
|
|
|
73 .. exception:: UnlockError
|
|
|
74
|
|
|
75 This is the base class for all exceptions raised when attempting to
|
|
|
76 unlock a file.
|
|
|
77
|
|
|
78 .. exception:: LockTimeout
|
|
|
79
|
|
|
80 This exception is raised if the :func:`LockFile.acquire` method is
|
|
|
81 called with a timeout which expires before an existing lock is released.
|
|
|
82
|
|
|
83 .. exception:: AlreadyLocked
|
|
|
84
|
|
|
85 This exception is raised if the :func:`LockFile.acquire` detects a
|
|
|
86 file is already locked when in non-blocking mode.
|
|
|
87
|
|
|
88 .. exception:: LockFailed
|
|
|
89
|
|
|
90 This exception is raised if the :func:`LockFile.acquire` detects some
|
|
|
91 other condition (such as a non-writable directory) which prevents it from
|
|
|
92 creating its lock file.
|
|
|
93
|
|
|
94 .. exception:: NotLocked
|
|
|
95
|
|
|
96 This exception is raised if the file is not locked when
|
|
|
97 :func:`LockFile.release` is called.
|
|
|
98
|
|
|
99 .. exception:: NotMyLock
|
|
|
100
|
|
|
101 This exception is raised if the file is locked by another thread or
|
|
|
102 process when :func:`LockFile.release` is called.
|
|
|
103
|
|
|
104 The following classes are provided:
|
|
|
105
|
|
|
106 .. class:: linklockfile.LinkLockFile(path, threaded=True)
|
|
|
107
|
|
|
108 This class uses the :func:`link(2)` system call as the basic lock
|
|
|
109 mechanism. *path* is an object in the file system to be locked. It need
|
|
|
110 not exist, but its directory must exist and be writable at the time the
|
|
|
111 :func:`acquire` and :func:`release` methods are called. *threaded* is
|
|
|
112 optional, but when set to :const:`True` locks will be distinguished
|
|
|
113 between threads in the same process.
|
|
|
114
|
|
|
115 .. class:: symlinklockfile.SymlinkLockFile(path, threaded=True)
|
|
|
116
|
|
|
117 This class uses the :func:`symlink(2)` system call as the basic lock
|
|
|
118 mechanism. The parameters have the same meaning and constraints as for
|
|
|
119 the :class:`LinkLockFile` class.
|
|
|
120
|
|
|
121 .. class:: mkdirlockfile.MkdirLockFile(path, threaded=True)
|
|
|
122
|
|
|
123 This class uses the :func:`mkdir(2)` system call as the basic lock
|
|
|
124 mechanism. The parameters have the same meaning and constraints as for
|
|
|
125 the :class:`LinkLockFile` class.
|
|
|
126
|
|
|
127 .. class:: sqlitelockfile.SQLiteLockFile(path, threaded=True)
|
|
|
128
|
|
|
129 This class uses the :mod:`sqlite3` module to implement the lock
|
|
|
130 mechanism. The parameters have the same meaning as for the
|
|
|
131 :class:`LinkLockFile` class.
|
|
|
132
|
|
|
133 .. class:: LockBase(path, threaded=True)
|
|
|
134
|
|
|
135 This is the base class for all concrete implementations and is available
|
|
|
136 at the lockfile package level so programmers can implement other locking
|
|
|
137 schemes.
|
|
|
138
|
|
|
139 .. function:: locked(path, timeout=None)
|
|
|
140
|
|
|
141 This function provides a decorator which insures the decorated function
|
|
|
142 is always called with the lock held.
|
|
|
143
|
|
|
144 By default, the :const:`LockFile` object refers to the
|
|
|
145 :class:`mkdirlockfile.MkdirLockFile` class on Windows. On all other
|
|
|
146 platforms it refers to the :class:`linklockfile.LinkLockFile` class.
|
|
|
147
|
|
|
148 When locking a file the :class:`linklockfile.LinkLockFile` class creates a
|
|
|
149 uniquely named hard link to an empty lock file. That hard link contains the
|
|
|
150 hostname, process id, and if locks between threads are distinguished, the
|
|
|
151 thread identifier. For example, if you want to lock access to a file named
|
|
|
152 "README", the lock file is named "README.lock". With per-thread locks
|
|
|
153 enabled the hard link is named HOSTNAME-THREADID-PID. With only per-process
|
|
|
154 locks enabled the hard link is named HOSTNAME--PID.
|
|
|
155
|
|
|
156 When using the :class:`mkdirlockfile.MkdirLockFile` class the lock file is a
|
|
|
157 directory. Referring to the example above, README.lock will be a directory
|
|
|
158 and HOSTNAME-THREADID-PID will be an empty file within that directory.
|
|
|
159
|
|
|
160 .. seealso::
|
|
|
161
|
|
|
162 Module :mod:`msvcrt`
|
|
|
163 Provides the :func:`locking` function, the standard Windows way of
|
|
|
164 locking (parts of) a file.
|
|
|
165
|
|
|
166 Module :mod:`posixfile`
|
|
|
167 The deprecated (since Python 1.5) way of locking files on Posix systems.
|
|
|
168
|
|
|
169 Module :mod:`fcntl`
|
|
|
170 Provides the current best way to lock files on Unix systems
|
|
|
171 (:func:`lockf` and :func:`flock`).
|
|
|
172
|
|
|
173 LockFile Objects
|
|
|
174 ----------------
|
|
|
175
|
|
|
176 :class:`LockFile` objects support the `context manager` protocol used by the
|
|
|
177 statement:`with` statement. The timeout option is not supported when used in
|
|
|
178 this fashion. While support for timeouts could be implemented, there is no
|
|
|
179 support for handling the eventual :exc:`Timeout` exceptions raised by the
|
|
|
180 :func:`__enter__` method, so you would have to protect the `with` statement with
|
|
|
181 a `try` statement. The resulting construct would not be any simpler than just
|
|
|
182 using a `try` statement in the first place.
|
|
|
183
|
|
|
184 :class:`LockFile` has the following user-visible methods:
|
|
|
185
|
|
|
186 .. method:: LockFile.acquire(timeout=None)
|
|
|
187
|
|
|
188 Lock the file associated with the :class:`LockFile` object. If the
|
|
|
189 *timeout* is omitted or :const:`None` the caller will block until the
|
|
|
190 file is unlocked by the object currently holding the lock. If the
|
|
|
191 *timeout* is zero or a negative number the :exc:`AlreadyLocked` exception
|
|
|
192 will be raised if the file is currently locked by another process or
|
|
|
193 thread. If the *timeout* is positive, the caller will block for that
|
|
|
194 many seconds waiting for the lock to be released. If the lock is not
|
|
|
195 released within that period the :exc:`LockTimeout` exception will be
|
|
|
196 raised.
|
|
|
197
|
|
|
198 .. method:: LockFile.release()
|
|
|
199
|
|
|
200 Unlock the file associated with the :class:`LockFile` object. If the
|
|
|
201 file is not currently locked, the :exc:`NotLocked` exception is raised.
|
|
|
202 If the file is locked by another thread or process the :exc:`NotMyLock`
|
|
|
203 exception is raised.
|
|
|
204
|
|
|
205 .. method:: is_locked()
|
|
|
206
|
|
|
207 Return the status of the lock on the current file. If any process or
|
|
|
208 thread (including the current one) is locking the file, :const:`True` is
|
|
|
209 returned, otherwise :const:`False` is returned.
|
|
|
210
|
|
|
211 .. method:: break_lock()
|
|
|
212
|
|
|
213 If the file is currently locked, break it.
|
|
|
214
|
|
|
215 .. method:: i_am_locking()
|
|
|
216
|
|
|
217 Returns true if the caller holds the lock.
|
|
|
218
|
|
|
219 Examples
|
|
|
220 --------
|
|
|
221
|
|
|
222 This example is the "hello world" for the :mod:`lockfile` package::
|
|
|
223
|
|
|
224 from lockfile import LockFile
|
|
|
225 lock = LockFile("/some/file/or/other")
|
|
|
226 with lock:
|
|
|
227 print lock.path, 'is locked.'
|
|
|
228
|
|
|
229 To use this with Python 2.4, you can execute::
|
|
|
230
|
|
|
231 from lockfile import LockFile
|
|
|
232 lock = LockFile("/some/file/or/other")
|
|
|
233 lock.acquire()
|
|
|
234 print lock.path, 'is locked.'
|
|
|
235 lock.release()
|
|
|
236
|
|
|
237 If you don't want to wait forever, you might try::
|
|
|
238
|
|
|
239 from lockfile import LockFile
|
|
|
240 lock = LockFile("/some/file/or/other")
|
|
|
241 while not lock.i_am_locking():
|
|
|
242 try:
|
|
|
243 lock.acquire(timeout=60) # wait up to 60 seconds
|
|
|
244 except LockTimeout:
|
|
|
245 lock.break_lock()
|
|
|
246 lock.acquire()
|
|
|
247 print "I locked", lock.path
|
|
|
248 lock.release()
|
|
|
249
|
|
|
250 You can also insure that a lock is always held when appropriately decorated
|
|
|
251 functions are called::
|
|
|
252
|
|
|
253 from lockfile import locked
|
|
|
254 @locked("/tmp/mylock")
|
|
|
255 def func(a, b):
|
|
|
256 return a + b
|
|
|
257
|
|
|
258 Other Libraries
|
|
|
259 ---------------
|
|
|
260
|
|
|
261 The idea of implementing advisory locking with a standard API is not new
|
|
|
262 with :mod:`lockfile`. There are a number of other libraries available:
|
|
|
263
|
|
|
264 * locknix - http://pypi.python.org/pypi/locknix - Unix only
|
|
|
265 * mx.MiscLockFile - from Marc André Lemburg, part of the mx.Base
|
|
|
266 distribution - cross-platform.
|
|
|
267 * Twisted - http://twistedmatrix.com/trac/browser/trunk/twisted/python/lockfile.py
|
|
|
268 * zc.lockfile - http://pypi.python.org/pypi/zc.lockfile
|
|
|
269
|
|
|
270
|
|
|
271 Contacting the Author
|
|
|
272 ---------------------
|
|
|
273
|
|
|
274 If you encounter any problems with ``lockfile``, would like help or want to
|
|
|
275 submit a patch, check http://launchpad.net/pylockfile
|
