๓ Vพ๗Xc@`sาdZddlmZddlmZddlmZddlZddlZddlZddlZddl m Z ddl m Z ej dd ƒZd „Zd „Zd „Zd „Zd„Zd„ZdS(s|System for specifying garbage collection (GC) of path based data. This framework allows for GC of data specified by path names, for example files on disk. gc.Path objects each represent a single item stored at a path and may be a base directory, /tmp/exports/0/... /tmp/exports/1/... ... or a fully qualified file, /tmp/train-1.ckpt /tmp/train-2.ckpt ... A gc filter function takes and returns a list of gc.Path items. Filter functions are responsible for selecting Path items for preservation or deletion. Note that functions should always return a sorted list. For example, base_dir = "/tmp" # create the directories for e in xrange(10): os.mkdir("%s/%d" % (base_dir, e), 0o755) # create a simple parser that pulls the export_version from the directory def parser(path): match = re.match("^" + base_dir + "/(\d+)$", path.path) if not match: return None return path._replace(export_version=int(match.group(1))) path_list = gc.get_paths("/tmp", parser) # contains all ten Paths every_fifth = gc.mod_export_version(5) print every_fifth(path_list) # shows ["/tmp/0", "/tmp/5"] largest_three = gc.largest_export_versions(3) print largest_three(all_paths) # shows ["/tmp/7", "/tmp/8", "/tmp/9"] both = gc.union(every_fifth, largest_three) print both(all_paths) # shows ["/tmp/0", "/tmp/5", # "/tmp/7", "/tmp/8", "/tmp/9"] # delete everything not in 'both' to_delete = gc.negation(both) for p in to_delete(all_paths): gfile.DeleteRecursively(p.path) # deletes: "/tmp/1", "/tmp/2", # "/tmp/3", "/tmp/4", "/tmp/6", i(tabsolute_import(tdivision(tprint_functionN(txrange(tgfiletPathspath export_versionc`s‡fd†}|S(sซCreates a filter that keeps the largest n export versions. Args: n: number of versions to keep. Returns: A filter function that keeps the n largest paths. c`s‡g}xEt|ƒD]7\}}|jdk rtj||j|fƒqqWgtjˆ|ƒD]\}}||^qa}t|ƒS(N(t enumeratetexport_versiontNonetheapqtheappushtnlargesttsorted(tpathstheaptidxtpatht_titkeepers(tn(sf/tmp/pip-build-h1VYrz/tensorflow/tensorflow-1.0.1.data/purelib/tensorflow/contrib/session_bundle/gc.pytkeepXs  /((RR((Rsf/tmp/pip-build-h1VYrz/tensorflow/tensorflow-1.0.1.data/purelib/tensorflow/contrib/session_bundle/gc.pytlargest_export_versionsOs c`s‡fd†}|S(sCreates a filter that keeps one of every n export versions. Args: n: interval size. Returns: A filter function that keeps exactly one path from each interval [0, n], (n, 2n], (2n, 3n], etc... If more than one path exists in an interval the largest is kept. c`si}x„|D]|}|jdkr(q n|jrKtj|jdˆƒnd}|j|dƒ}| s||j|jkr |||>s