=========================================== kissdx scaled picture cache – Specification =========================================== BACKGROUND Picture scaling has been added to kissdx, but it's slower than it needs be, because the scaling is performed twice (due to the fact that the KiSS player connects twice in the process of getting the picture). To reduce this to scaling only once, the scaled picture must be stored in a common memory block or on disk. Storing on disk is easy, and expanding on that, a cache of recently scaled pictures can be maintained. CONTROLLING THE CACHE A new config option will specify the maximum number of pictures to keep in this cache: picturecachesize = 10 The absence of this config option (or a value of 0) will disable the caching feature, and thus make picture scaling slower than necessary (see above). If picture scaling is enabled, a cache size of 1 is recommended instead of disabling the cache altogether. A cache that can at least hold the current picture eliminates the problem of scaling being performed twice, thus nearly doubling the display speed of each scaled picture. To enforce the cache size limit, the cache must be scanned to determine its current size and contents. This scanning is expensive performance-wise, so we will impose a limit on how often the scanning is performed, by maintaining a last-scanned timestamp in a hidden file inside the cache root directory (see below). .scaninfo A second cache-related config option will specify the limit on how often the cache should be scanned and trimmed: picturecachetrimminginterval = 5 In the special case of a picturecachesize = 1, this trimming interval will not be respected: The cache will be trimmed every time a new picture is to be added, i.e. the previously cached picture will always be removed from the cache before the current picture is added. IMPLEMENTATION OF THE CACHE The picture cache will reside in a hidden directory under the Pictures directory, which we will call the cache root: .scaledpics The cache will consist of a set of entries, one entry per scaled picture. Each entry will consist of 1. One subdirectory with the name = the base name of the original picture file. The cache entries for all picture files sharing the same base name will be in this subdirectory, which we will call a cache segment. 2. One file containing The scaled picture. Filename: / E.g. IMG012345.jpeg/03984834 Contents: This file contains the image data of the scaled picture. 3. One file containing information about the picture. Filename: /.info E.g. IMG012345.jpeg/03984834.info The of the info file will be the same as the of the scaled picture file, creating the logical link between the two files. Contents: 1. Complete path/filename/extension of original picture file 2. Timestamp of when scaling took place 3. Picture scaling settings that were used to produce the scaled picture file (width, height, zoom) There will be no index file or other supporting files for the cache. This will keep the implementation simple, utilizing the host's file system instead. The downside of using multiple info files instead of a single index file will be degraded performance for a large cache, but this effect should be reduced by having a unique subdirectory for each base name, giving quick access to a presumably small segment of the cache. OPERATION OF THE CACHE When a request comes in for a picture that is to be scaled, a look-up will be made to see if the file is already in the cache. This look-up consists of 1. locating the appropriate subdirectory for the picture file's base name (i.e. the cache segment that potentially contains the cache entry for that picture). 2. opening all the info files in that subdirectory one by one, checking the complete original path/filename inside, to see if that matches the requested file. If we find a matching filename, we also verify that the cached file was produced with the current scaling settings and zoom, and that the original picture file has not been modified since it was last cached. If any of these tests fail, we remove the entry from the cache and we have no match. This will force the picture to be scaled and cached anew. If a match is found, the scaled picture file corresponding to the matching info file will be served from the cache in the response to this request. If no match is found in the cache, the requested file will be scaled and placed in the cache, and finally it will be served to the player from the cache. Placing a new file in the cache involves 1. Making room for the new file: If the cache has reached its maximum number of entries as specified by the picturecachesize config option, the least recently added entry will be removed from the cache. This can be done by a directory scan of the .scaledpics directory and its subdirectories, finding the info file that has the oldest "modify" timestamp. This info file and its corresponding scaled picture file will then be deleted. We can use ftw() or nftw() for the directory + subdirectory scanning. 2. Saving the new file: a. If it does not exist already, the cache root directory is created with a name of .scaledpics in the Pictures directory. b. If it does not exist already, a new subdirectory of .scaledpics is created with a name = the base name of the requested picture file. c. A new unique filename is generated inside that subdirectory, and the scaled picture file will be created with that name. We will use mkstemp() or similar to create the new file. d. A new info file will be created in the same subdirectory, with the same filename and the .info extension. Into this file we write the original path etc. as described above. PICTURE FORMATS The cache should include all supported picture types, but currently only jpeg will be supported, since we only scale jpegs. Is there a libpng, a libbmp, etc.? MULTI-CLIENT CONSIDERATIONS kissdx supports simultaneous access by multiple KiSS players. This is quite easy to accomplish as long as no shared state information is being kept on the server. This new cache is shared state information, and will be governed by the following rules: 1. There will be only one cache that all clients have in common. 2. As far as feasible, the code must be designed to avoid deadlocks or errors due to concurrent use of the cache by multiple clients. STATUS Implemented in kissdx 0.13-6.