Package entropy :: Module tools

Module tools

source code


Author: Fabio Erculiani <[email protected]>

Contact: [email protected]

Copyright: Fabio Erculiani

License: GPL-2

Entropy miscellaneous tools module. In this module are enclosed all the miscellaneous functions used around the Entropy codebase.

Functions
bool
is_root()
Return whether running process has root priviledges.
source code
bool
is_user_in_entropy_group(uid=None)
Return whether UID or given UID (through uid keyword argument) is in the "entropy" group (see entropy.const.etpConst['sysgroup']).
source code
int
get_uid_from_user(username)
Return UID for given username or -1 if not available.
source code
int
get_gid_from_group(groupname)
Return GID value for given system group name if exists, otherwise return -1.
source code
string or None
get_user_from_uid(uid)
Return username belonging to given system UID.
source code
string or None
get_group_from_gid(gid)
Return group name belonging to given system GID
source code
 
kill_threads()
Call entropy.const's const_kill_threads() method.
source code
 
print_traceback(f=None)
Function called by Entropy when an exception occurs with the aim to give user a clue of what went wrong.
source code
string
get_traceback(tb_obj=None)
Return last available Python traceback.
source code
list of strings
print_exception(silent=False, tb_data=None, all_frame_data=False)
Print last Python exception and frame variables values (if available) to stdout.
source code
string or bool
get_remote_data(url, timeout=5)
Fetch data at given URL (all the ones supported by Python urllib) and return it.
source code
bool
is_supported_image_file(path)
Return whether passed image file path "path" references a valid image file.
source code
bool
is_april_first()
Return whether today is April, 1st.
source code
bool
is_xmas()
Return whether today is April, 1st.
source code
bool
is_author_bday()
Return whether today is lxnay's birthday.
source code
bool
is_st_valentine()
Return whether today is April, 1st.
source code
 
add_proxy_opener(module, data)
Add proxy opener to urllib module.
source code
bool
is_valid_ascii(string)
Return whether passed string only contains valid ASCII characters.
source code
bool
is_valid_unicode(string)
Return whether passed string is unicode.
source code
bool
is_valid_email(email)
Return whether passed string is contains a valid email address.
source code
bool
islive()
Return whether System is running in Live mode (off a CD/DVD).
source code
int
get_file_size(file_path)
Return size of given path passed in "file_path".
source code
int
sum_file_sizes(file_list)
Return file size sum of given list of paths.
source code
int
sum_file_sizes_hardlinks(file_list)
Return file size sum of given list of paths.
source code
bool
check_required_space(mountpoint, bytes_required)
Check available space in mount point and if it satisfies the amount of required bytes given.
source code
 
getstatusoutput(cmd)
Return (status, output) of executing cmd in a shell.
source code
bool
movefile(src, dest, src_basedir=None)
Move a file from source to destination in an atomic way.
source code
 
rename_keep_permissions(src, dest)
Call rename() for src -> dest files keeping dest permission bits and ownership.
source code
 
atomic_write(filepath, content_str, encoding)
Atomically write string at content_str using given encoding to file.
source code
int
get_random_number()
Return a random number between 10000 and 99999.
source code
list
split_indexable_into_chunks(mystr, chunk_len)
Split indexable object into chunks.
source code
string
md5sum(filepath)
Calculate md5 hash of given file at path.
source code
string
sha512(filepath)
Calculate SHA512 hash of given file at path.
source code
string
sha256(filepath)
Calculate SHA256 hash of given file at path.
source code
string
sha1(filepath)
Calculate SHA1 hash of given file at path.
source code
string
md5sum_directory(directory)
Return md5 hex digest of files in given directory
source code
hashlib.md5
md5obj_directory(directory)
Return hashlib.md5 instance of calculated md5 of files in given directory
source code
 
uncompress_file(file_path, destination_path, opener)
Uncompress file at file_path into destination_path using file opener function passed.
source code
 
compress_file(file_path, destination_path, opener, compress_level=None)
Compress file at file_path into destination_path (file path) using transparent compression file opener and given compression level (from 0 to 9).
source code
 
compress_files(dest_file, files_to_compress, compressor='bz2')
Compress file paths listed inside files_to_compress into dest_file using given compression type "compressor".
source code
 
universal_uncompress(compressed_file, dest_path, catch_empty=False)
Universally uncompress (automatic detection) compressed file at compressed_file into dest_path.
source code
int
get_uncompressed_size(compressed_file)
Return the size of uncompressed data of a tarball (compression algos that tarfile supports).
source code
string
unpack_gzip(gzipfilepath)
Unpack .gz file.
source code
string
unpack_bzip2(bzip2filepath)
Unpack .bz2 file.
source code
string
generate_entropy_delta_file_name(pkg_name_a, pkg_name_b, hash_tag)
Generate Entropy package binary delta file name basing on package file names given (from pkg_path_a to pkg_path_b).
source code
bool
is_entropy_delta_available()
Return whether Entropy delta packages support is enabled by checking if bsdiff executables are available.
source code
string or None
generate_entropy_delta(pkg_path_a, pkg_path_b, hash_tag, pkg_compression=None)
Generate Entropy package delta between pkg_path_a (from file) and pkg_path_b (to file).
source code
 
apply_entropy_delta(pkg_path_a, delta_path, new_pkg_path_b, pkg_compression=None)
Apply Entropy package delta file to pkg_path_a generating pkg_path_b (which is returned in case of success).
source code
 
aggregate_entropy_metadata(entropy_package_file, entropy_metadata_file)
Add Entropy metadata dump file to given Entropy package file.
source code
bool
dump_entropy_metadata(entropy_package_file, entropy_metadata_file)
Dump Entropy package metadata from Entropy package file to entropy_metadata_file
source code
bool
remove_entropy_metadata(entropy_package_file, save_path)
Remove Entropy metadata from Entropy package file.
source code
string
create_md5_file(filepath)
Create valid MD5 file off filepath.
source code
string
create_sha512_file(filepath)
Create valid SHA512 file off filepath.
source code
string
create_sha256_file(filepath)
Create valid SHA256 file off filepath.
source code
string
create_sha1_file(filepath)
Create valid SHA1 file off filepath.
source code
bool
compare_md5(filepath, checksum)
Compare MD5 of filepath with the one given (checksum).
source code
string
get_hash_from_md5file(md5path)
Extract md5 hash from md5 file.
source code
bool
compare_sha512(filepath, checksum)
Compare SHA512 of filepath with the one given (checksum).
source code
bool
compare_sha256(filepath, checksum)
Compare SHA256 of filepath with the one given (checksum).
source code
bool
compare_sha1(filepath, checksum)
Compare SHA1 of filepath with the one given (checksum).
source code
string
md5string(string)
Return md5 hex digest of given string
source code
list
generic_file_content_parser(filepath, comment_tag='#', filter_comments=True, encoding=None)
Generic unix-style file content parser.
source code
bool
isnumber(x)
Determine whether x is a number of any sort.
source code
bool
istextfile(filename, blocksize=512)
Return whether file at filename is a text file by reading the first blocksize bytes.
source code
bool
istext(mystring)
Determine whether given string is text.
source code
urllib.parse
spliturl(url)
Split any URL (ftp, file, http) into separate entities using urllib Python module.
source code
bool
is_valid_uri(url)
Determine whether given url string is a valid URI, this function internally calls spliturl and looks for a set scheme.
source code
 
apply_tarball_ownership(filepath, prefix_path)
Given an already extracted tarball available at prefix_path, and the original tarball file path at filepath, apply files and directories ownership to belonged files in prefix_path looking at tar metadata.
source code
int
uncompress_tarball(filepath, extract_path=None, catch_empty=False)
Unpack tarball file (supported compression algorithm is given by tarfile module) respecting directory structure, mtime and permissions.
source code
string
bytes_into_human(xbytes)
Convert byte size into human readable format.
source code
string
convert_unix_time_to_human_time(unixtime)
Convert UNIX time (int) into human readable time format.
source code
string
get_year()
Return current year string.
source code
string
convert_seconds_to_fancy_output(seconds)
Convert seconds (int) into a more fancy and human readable output.
source code
bool
write_parameter_to_file(config_file, name, data)
Write configuration file parameter to file.
source code
tuple
extract_setting(raw_line)
Extract configuration file setting key and value from string representing a configuration file line.
source code
bool or None
setting_to_bool(setting)
Convert entropy setting string which should represent a bool setting into a bool type, if possible, otherwise return None.
source code
bool or None
setting_to_int(setting, lower_bound, upper_bound)
Convert entropy setting string which should represent a int setting into a int type, if possible, otherwise return None.
source code
string or None
expand_plain_package_mirror(mirror, product, repository_id)
Expand plain mirror URL adding product and repository identifier data to it.
source code
string or None
expand_plain_database_mirror(mirror, product, repository_id, branch)
Expand plain database mirror URL adding product, repository identifier and branch data to it.
source code
bool
validate_repository_id(repository_id)
Validate Entropy repository identifier string.
source code
bool
validate_package_name(package_name)
Validate Entropy package name string.
source code
bool
validate_branch_name(branch)
Validate Entropy branch name string.
source code
bool
is_entropy_package_file(entropy_package_path)
Determine whether given package path is a valid Entropy package file.
source code
bool
is_valid_string(string)
Return whether given string only contains ASCII printable chars (from 0x20 to 0xFF).
source code
bool
is_valid_path_string(path)
Return whether given path is a valid path string (whitelisting valid characters).
source code
bool
is_valid_path(path)
Return whether given path is valid (it uses os.stat()).
source code
bool
is_valid_md5(string)
Return whether given string is a valid md5 hex digest.
source code
int
elf_class_strtoint(elf_class_str)
Convert an ELF class metadataum string to its int value.
source code
int
read_elf_class(elf_file)
Read ELF class metadatum from ELF file.
source code
bool
is_elf_file(elf_file)
Determine whether given file path points to an ELF file object.
source code
list
parse_rpath(rpath)
Parse RPATH metadata stored in repository and return an ordered list of paths.
source code
string
resolve_dynamic_library(library, requiring_executable)
Resolve given library name (as contained into ELF metadata) to a library path.
source code
set
read_elf_dynamic_libraries(elf_file)
Extract NEEDED metadatum from ELF file at path.
source code
dict or None
read_elf_metadata(elf_file)
Extract soname, elf class, runpath and NEEDED metadata from ELF file.
source code
set
read_elf_real_dynamic_libraries(elf_file)
This function is similar to read_elf_dynamic_libraries but uses ldd to retrieve a list of "real" .so library dependencies used by the ELF file.
source code
set
read_elf_broken_symbols(elf_file)
Extract broken symbols from ELF file.
source code
list
read_elf_linker_paths(elf_file)
Extract built-in linker paths (RUNPATH and RPATH) from ELF file.
source code
string
xml_from_dict_extended(dictionary)
Serialize a simple dict object into an XML string.
source code
dict
dict_from_xml_extended(xml_string)
Deserialize an XML string representing a dict object back into a dict object.
source code
string
xml_from_dict(dictionary)
Serialize a dict object into a "simple" XML string.
source code
dict
dict_from_xml(xml_string)
Deserialize an XML string representing a dict (created by xml_from_dict) back into a dict object.
source code
tuple
collect_linker_paths()
Collect dynamic linker paths set into /etc/ld.so.conf.
source code
list
collect_paths()
Return env var PATH value split using ":" as separator.
source code
string
create_package_dirpath(branch, nonfree=False, restricted=False)
Create Entropy package relative directory path used for building EntropyRepository "download" metadatum and for handling package file life by Entropy Server.
source code
 
recursive_directory_relative_listing(empty_list, base_directory, _nested=False)
Takes an array(list) and appends all files from dir down the directory tree.
source code
List
flatten(mylist)
Recursively traverse nested lists and return a single list containing all non-list elements that are found.
source code
 
codecs_fdopen(fd, mode, encoding, errors='strict')
Copycats codecs.open() but accepts fd (file descriptors) as input file handle.
source code
int
total_memory()
Return the amount of total system memory in megabytes.
source code
Variables
  valid_path_regexp = re.compile(r'^([A-Za-z0-9/\.:-_]+)$')
  __package__ = 'entropy'
Function Details

is_root()

source code 

Return whether running process has root priviledges.

Returns: bool
root priviledges

is_user_in_entropy_group(uid=None)

source code 

Return whether UID or given UID (through uid keyword argument) is in the "entropy" group (see entropy.const.etpConst['sysgroup']).

Parameters:
  • uid (int) - valid system uid
Returns: bool
True, if UID is in the "entropy" group

get_uid_from_user(username)

source code 

Return UID for given username or -1 if not available.

Parameters:
  • username (string) - valid system username
Returns: int
UID if username is valid, otherwise -1

get_gid_from_group(groupname)

source code 

Return GID value for given system group name if exists, otherwise return -1.

Parameters:
  • groupname (string) - valid system group
Returns: int
resolved GID or -1 if not available

get_user_from_uid(uid)

source code 

Return username belonging to given system UID.

Parameters:
  • uid (int) - valid system UID
Returns: string or None
username

get_group_from_gid(gid)

source code 

Return group name belonging to given system GID

Parameters:
  • gid (int) - valid system GID
Returns: string or None
group name

kill_threads()

source code 

Call entropy.const's const_kill_threads() method. Service function available also here.

print_traceback(f=None)

source code 

Function called by Entropy when an exception occurs with the aim to give user a clue of what went wrong.

Parameters:
  • f (valid file handle) - write to f (file) object instead of stdout

get_traceback(tb_obj=None)

source code 

Return last available Python traceback.

Parameters:
  • tb_obj (Python traceback instance) - Python traceback object
Returns: string
traceback data

print_exception(silent=False, tb_data=None, all_frame_data=False)

source code 

Print last Python exception and frame variables values (if available) to stdout.

Parameters:
  • silent (bool) - do not print to stdout
  • tb_data (Python traceback instance) - Python traceback object
  • all_frame_data (bool) - print all variables in every frame
Returns: list of strings
exception data

get_remote_data(url, timeout=5)

source code 

Fetch data at given URL (all the ones supported by Python urllib) and return it.

Parameters:
  • url (string) - URL string
  • timeout (int) - fetch timeout in seconds
Returns: string or bool
fetched data or False (when error occurred)

is_supported_image_file(path)

source code 

Return whether passed image file path "path" references a valid image file. Currently supported image file types are: PNG, JPEG, BMP, GIF.

Parameters:
  • path (string) - path pointing to a possibly valid image file
Returns: bool
True if path references a valid image file

is_april_first()

source code 

Return whether today is April, 1st. Please keep the joke.

Returns: bool
True if April 1st

is_xmas()

source code 

Return whether today is April, 1st. Please keep the joke.

Returns: bool
True if April 1st

is_author_bday()

source code 

Return whether today is lxnay's birthday.

Returns: bool
True if November 15

is_st_valentine()

source code 

Return whether today is April, 1st. Please keep the joke.

Returns: bool
True if April 1st

add_proxy_opener(module, data)

source code 

Add proxy opener to urllib module.

Parameters:
  • module (Python module) - urllib module
  • data (dict) - proxy settings

is_valid_ascii(string)

source code 

Return whether passed string only contains valid ASCII characters.

Parameters:
  • string (string) - string to test
Returns: bool
True if string contains pure ASCII

is_valid_unicode(string)

source code 

Return whether passed string is unicode.

Parameters:
  • string (string) - string to test
Returns: bool
True if string is unicode

is_valid_email(email)

source code 

Return whether passed string is contains a valid email address.

Parameters:
  • email (string) - string to test
Returns: bool
True if string is a valid email

islive()

source code 

Return whether System is running in Live mode (off a CD/DVD). See entropy.const.const_islive() for more information.

Returns: bool
True if System is running in Live mode

get_file_size(file_path)

source code 

Return size of given path passed in "file_path".

Parameters:
  • file_path (string) - path to an existing file
Returns: int
file size in bytes
Raises:
  • OSError - if file referenced in file_path is not available

sum_file_sizes(file_list)

source code 

Return file size sum of given list of paths. NOTE: This function does NOT consider hardlinks, roughly summing up file_list elements.

Parameters:
  • file_list (list) - list of file paths
Returns: int
summed size in bytes

sum_file_sizes_hardlinks(file_list)

source code 

Return file size sum of given list of paths. NOTE: This function does consider hardlinks, not counting the same files more than once.

Parameters:
  • file_list (list) - list of file paths
Returns: int
summed size in bytes

check_required_space(mountpoint, bytes_required)

source code 

Check available space in mount point and if it satisfies the amount of required bytes given.

Parameters:
  • mountpoint (string) - mount point
  • bytes_required (bool) - amount of bytes required to make function return True
Returns: bool
if True, required space is available

movefile(src, dest, src_basedir=None)

source code 

Move a file from source to destination in an atomic way.

Parameters:
  • src (string) - source path
  • dest (string) - destination path
  • src_basedir (string) - source path base directory, used to properly handle symlink under certain circumstances
Returns: bool
True, if file was moved successfully

rename_keep_permissions(src, dest)

source code 

Call rename() for src -> dest files keeping dest permission bits and ownership. Useful in combination with mkstemp() If dest doesn't exist, ownership and permissions will be set through entropy.const's const_secure_config_file(). File is moved using entropy.tools.movefile()

Parameters:
  • src (string) - path to source file
  • dest (string) - path to dest file
Raises:
  • OSError - if file cannot be moved.

atomic_write(filepath, content_str, encoding)

source code 

Atomically write string at content_str using given encoding to file.

Parameters:
  • filepath (string) - path where to write data atomically
  • content_str (string) - string to write
  • encoding (string) - encoding to use
Raises:
  • IOError - if data cannot be written
  • OSError - same as above

get_random_number()

source code 

Return a random number between 10000 and 99999.

Returns: int
random number

split_indexable_into_chunks(mystr, chunk_len)

source code 

Split indexable object into chunks.

Parameters:
  • mystr (Python object) - indexable object
  • chunk_len (int) - maximum length of a single chunk
Returns: list
list of chunks

md5sum(filepath)

source code 

Calculate md5 hash of given file at path.

Parameters:
  • filepath (string) - path to file
Returns: string
md5 hex digest

sha512(filepath)

source code 

Calculate SHA512 hash of given file at path.

Parameters:
  • filepath (string) - path to file
Returns: string
SHA512 hex digest

sha256(filepath)

source code 

Calculate SHA256 hash of given file at path.

Parameters:
  • filepath (string) - path to file
Returns: string
SHA256 hex digest

sha1(filepath)

source code 

Calculate SHA1 hash of given file at path.

Parameters:
  • filepath (string) - path to file
Returns: string
SHA1 hex digest

md5sum_directory(directory)

source code 

Return md5 hex digest of files in given directory

Parameters:
  • directory (string) - path to directory
Returns: string
md5 hex digest

md5obj_directory(directory)

source code 

Return hashlib.md5 instance of calculated md5 of files in given directory

Parameters:
  • directory (string) - path to directory
Returns: hashlib.md5
hashlib.md5 instance

uncompress_file(file_path, destination_path, opener)

source code 

Uncompress file at file_path into destination_path using file opener function passed.

Parameters:
  • file_path (string) - path to file
  • destination_path (string) - destination path
  • opener (function) - file_path opener function

compress_file(file_path, destination_path, opener, compress_level=None)

source code 

Compress file at file_path into destination_path (file path) using transparent compression file opener and given compression level (from 0 to 9).

Parameters:
  • file_path (string) - path to compress
  • destination_path (string) - path where to save compressed file
  • opener (function) - compressed file_path open function
  • compress_level (int) - compression level, from 0 to 9

compress_files(dest_file, files_to_compress, compressor='bz2')

source code 

Compress file paths listed inside files_to_compress into dest_file using given compression type "compressor". Supported compression types are "bz2" and "gz".

Parameters:
  • dest_file (string) - path where to save compressed file
  • files_to_compress (list) - list of file paths to compress
  • compressor (string) - compressor type
Raises:
  • AttributeError - if compressor value is unsupported

universal_uncompress(compressed_file, dest_path, catch_empty=False)

source code 

Universally uncompress (automatic detection) compressed file at compressed_file into dest_path. "catch_empty" is used in case of empty compressed files, in which case a tarfile.ReadError exception is raised.

Parameters:
  • compressed_file (string) - path to compressed file
  • dest_path (string) - path where to uncompress compressed file content
  • catch_empty (bool) - if True, empty compressed file won't cause tarfile.ReadError exception to be raised

get_uncompressed_size(compressed_file)

source code 

Return the size of uncompressed data of a tarball (compression algos that tarfile supports).

Parameters:
  • compressed_file (string) - path to compressed file
Returns: int
size of the data inside the tarball

unpack_gzip(gzipfilepath)

source code 

Unpack .gz file.

Parameters:
  • gzipfilepath (string) - path to .gz file
Returns: string
path to uncompressed file

unpack_bzip2(bzip2filepath)

source code 

Unpack .bz2 file.

Parameters:
  • bzip2filepath (string) - path to .bz2 file
Returns: string
path to uncompressed file

generate_entropy_delta_file_name(pkg_name_a, pkg_name_b, hash_tag)

source code 

Generate Entropy package binary delta file name basing on package file names given (from pkg_path_a to pkg_path_b). hash_tag is by convention an md5 hash

Parameters:
  • pkg_name_a (string) - package file name A
  • pkg_name_b (string) - package file name B
  • hash_tag (string) - arbitrary hash tag appended to file name
Returns: string
package delta file name (not full path!)
Raises:
  • AttributeError - if api is unsupported

is_entropy_delta_available()

source code 

Return whether Entropy delta packages support is enabled by checking if bsdiff executables are available. Moreover, if ETP_NO_EDELTA environment variable is set, this function will return False.

Returns: bool
True, if service is available

generate_entropy_delta(pkg_path_a, pkg_path_b, hash_tag, pkg_compression=None)

source code 

Generate Entropy package delta between pkg_path_a (from file) and pkg_path_b (to file).

Parameters:
  • pkg_path_a (string) - package path A (from file)
  • pkg_path_a (string) - package path B (to file)
  • hash_tag (string) - hash tag to append to Entropy package delta file name
  • pkg_compression - default package compression, can be "bz2" or "gz". if None, "bz2" is selected.
Returns: string or None
path to newly created delta file, return None if error
Raises:
  • KeyError - if pkg_compression is unsupported
  • IOError - if delta cannot be generated
  • OSError - if some other error happens during the generation

apply_entropy_delta(pkg_path_a, delta_path, new_pkg_path_b, pkg_compression=None)

source code 

Apply Entropy package delta file to pkg_path_a generating pkg_path_b (which is returned in case of success). If delta cannot be generated, IOError is raised.

Parameters:
  • pkg_path_a (string) - path to package A
  • delta_path (string) - path to entropy package delta
  • new_pkg_path_b (string) - path where to store newly created package B
  • pkg_compression - default package compression, can be "bz2" or "gz". if None, "bz2" is selected.
Raises:
  • IOError - if delta cannot be generated.

aggregate_entropy_metadata(entropy_package_file, entropy_metadata_file)

source code 

Add Entropy metadata dump file to given Entropy package file.

Parameters:
  • entropy_package_file (string) - path to Entropy package file
  • entropy_metadata_file (string) - path to Entropy metadata file

dump_entropy_metadata(entropy_package_file, entropy_metadata_file)

source code 

Dump Entropy package metadata from Entropy package file to entropy_metadata_file

Parameters:
  • entropy_package_file (string) - path to Entropy package file
  • entropy_metadata_file (string) - path where to store extracted metadata
Returns: bool
True, if extraction went successful

remove_entropy_metadata(entropy_package_file, save_path)

source code 

Remove Entropy metadata from Entropy package file. Save new Entropy package file into save_path.

Parameters:
  • entropy_package_file (string) - path to Entropy package file
  • save_path (string) - path where to save new "Entropy" package file (without Entropy metadata)
Returns: bool
True, if removal went successful

create_md5_file(filepath)

source code 

Create valid MD5 file off filepath.

Parameters:
  • filepath (string) - file path to read
Returns: string
path to MD5 file

create_sha512_file(filepath)

source code 

Create valid SHA512 file off filepath.

Parameters:
  • filepath (string) - file path to read
Returns: string
path to SHA512 file

create_sha256_file(filepath)

source code 

Create valid SHA256 file off filepath.

Parameters:
  • filepath (string) - file path to read
Returns: string
path to SHA256 file

create_sha1_file(filepath)

source code 

Create valid SHA1 file off filepath.

Parameters:
  • filepath (string) - file path to read
Returns: string
path to SHA1 file

compare_md5(filepath, checksum)

source code 

Compare MD5 of filepath with the one given (checksum).

Parameters:
  • filepath (string) - path to file to "md5sum"
  • checksum (string) - known to be good MD5 checksum
Returns: bool
True, if MD5 matches

get_hash_from_md5file(md5path)

source code 

Extract md5 hash from md5 file. If md5 file is corrupted or invalid, raise ValueError.

Parameters:
  • md5path (string) - path to .md5 file
Returns: string
md5 hex digest
Raises:
  • ValueError - if md5path contains invalid data

compare_sha512(filepath, checksum)

source code 

Compare SHA512 of filepath with the one given (checksum).

Parameters:
  • filepath (string) - path to file to check
  • checksum (string) - known to be good SHA512 checksum
Returns: bool
True, if SHA512 matches

compare_sha256(filepath, checksum)

source code 

Compare SHA256 of filepath with the one given (checksum).

Parameters:
  • filepath (string) - path to file to check
  • checksum (string) - known to be good SHA256 checksum
Returns: bool
True, if SHA256 matches

compare_sha1(filepath, checksum)

source code 

Compare SHA1 of filepath with the one given (checksum).

Parameters:
  • filepath (string) - path to file to check
  • checksum (string) - known to be good SHA1 checksum
Returns: bool
True, if SHA1 matches

md5string(string)

source code 

Return md5 hex digest of given string

Parameters:
  • string (string) - string to "md5"
Returns: string
md5 hex digest

generic_file_content_parser(filepath, comment_tag='#', filter_comments=True, encoding=None)

source code 

Generic unix-style file content parser. Return a list of parsed lines with filtered comments.

Parameters:
  • filepath (string) - configuration file to parse
  • comment_tag (string) - default comment tag (column where comments starts) if line already contains valid data (doesn't start with comment_tag)
  • filter_comments (bool) - filter out comments, True by default. Are considered comments the lines starting with "#"
Returns: list
list representing file content

isnumber(x)

source code 

Determine whether x is a number of any sort. "x" can be a string or float.

Parameters:
  • x (Python object) - misterious object
Returns: bool
True, if x can be converted to int

istextfile(filename, blocksize=512)

source code 

Return whether file at filename is a text file by reading the first blocksize bytes.

Parameters:
  • filename (string) - file path to parse
  • blocksize (int) - chunk of bytes to read
Returns: bool
True, if text file

istext(mystring)

source code 

Determine whether given string is text.

Parameters:
  • mystring (string) - string to parse
Returns: bool
True, if string is text

spliturl(url)

source code 

Split any URL (ftp, file, http) into separate entities using urllib Python module.

Parameters:
  • url (string) - URL sto split
Returns: urllib.parse
urllib.parse instance

is_valid_uri(url)

source code 

Determine whether given url string is a valid URI, this function internally calls spliturl and looks for a set scheme. Anything that matches the string "something://" will be considered valid.

Parameters:
  • url (string) - URL sto split
Returns: bool
True if URI

apply_tarball_ownership(filepath, prefix_path)

source code 

Given an already extracted tarball available at prefix_path, and the original tarball file path at filepath, apply files and directories ownership to belonged files in prefix_path looking at tar metadata. This is required because users and groups referenced in tarballs are created at package setup phase during install.

uncompress_tarball(filepath, extract_path=None, catch_empty=False)

source code 

Unpack tarball file (supported compression algorithm is given by tarfile module) respecting directory structure, mtime and permissions.

Parameters:
  • filepath (string) - path to tarball file
  • extract_path (string) - path where to extract tarball
  • catch_empty (bool) - do not raise exceptions when trying to unpack empty file
Returns: int
exit status

bytes_into_human(xbytes)

source code 

Convert byte size into human readable format.

Parameters:
  • xbytes (int) - number of bytes
Returns: string
number of bytes in human readable format

convert_unix_time_to_human_time(unixtime)

source code 

Convert UNIX time (int) into human readable time format.

Parameters:
  • unixtime (int) - UNIX time
Returns: string
human readable time format

get_year()

source code 

Return current year string.

Returns: string
current year (20xx)

convert_seconds_to_fancy_output(seconds)

source code 

Convert seconds (int) into a more fancy and human readable output.

Parameters:
  • seconds (int) - number of seconds
Returns: string
human readable output

write_parameter_to_file(config_file, name, data)

source code 

Write configuration file parameter to file. name is used as key and data as value. Any older setting will be replaced. Disabled parameters won't be enabled (lines starting with "#").

Parameters:
  • config_file (string) - path to configuration file
  • name (string) - configuration parameter name
  • data (string) - configuration parameter value
Returns: bool
True, if executed properly

extract_setting(raw_line)

source code 

Extract configuration file setting key and value from string representing a configuration file line.

Parameters:
  • raw_line (string) - configuration file line
Returns: tuple
extracted setting key and value, if found, otherwise (None, None) if setting|key or setting=key is not found.

setting_to_bool(setting)

source code 

Convert entropy setting string which should represent a bool setting into a bool type, if possible, otherwise return None.

Parameters:
  • setting (string) - raw setting value that should represent a bool
Returns: bool or None
bool value, or None

setting_to_int(setting, lower_bound, upper_bound)

source code 

Convert entropy setting string which should represent a int setting into a int type, if possible, otherwise return None. Also check against lower and upper bounds, if different than None.

Parameters:
  • setting (string) - raw setting value that should represent a bool
Returns: bool or None
bool value, or None

expand_plain_package_mirror(mirror, product, repository_id)

source code 

Expand plain mirror URL adding product and repository identifier data to it.

Parameters:
  • mirror (string) - mirror URL
  • product (string) - Entropy repository product
  • repository_id (string) - repository identifier
Returns: string or None
expanded URL or None if invalid

expand_plain_database_mirror(mirror, product, repository_id, branch)

source code 

Expand plain database mirror URL adding product, repository identifier and branch data to it.

Parameters:
  • mirror (string) - mirror URL
  • product (string) - Entropy repository product
  • repository_id (string) - repository identifier
Returns: string or None
expanded URL or None if invalid

validate_repository_id(repository_id)

source code 

Validate Entropy repository identifier string.

Parameters:
  • repository_id (string) - entropy repository identifier
Returns: bool
True if repository_id is a valid string, False otherwise

validate_package_name(package_name)

source code 

Validate Entropy package name string.

@param package_name; the actual package name (either key or version)

Parameters:
  • package_name (string)
Returns: bool
True, if package_name is a valid package name

validate_branch_name(branch)

source code 

Validate Entropy branch name string.

@param branch; the actual branch name

Parameters:
  • branch (string)
Returns: bool
True, if branch is valid

is_entropy_package_file(entropy_package_path)

source code 

Determine whether given package path is a valid Entropy package file.

Parameters:
  • entropy_package_path (string) - path to Entropy package file
Returns: bool
True, if valid

is_valid_string(string)

source code 

Return whether given string only contains ASCII printable chars (from 0x20 to 0xFF).

Parameters:
  • string (string) - string to test
Returns: bool
True, if valid

is_valid_path_string(path)

source code 

Return whether given path is a valid path string (whitelisting valid characters). Regexp is ^([A-Za-z0-9/\.:-]+)$ and only works with ASCII paths.

Parameters:
  • path (string) - path to test
Returns: bool
True, if valid

is_valid_path(path)

source code 

Return whether given path is valid (it uses os.stat()). Broken symlinks will return False.

Parameters:
  • path (string) - path to test
Returns: bool
True, if valid

is_valid_md5(string)

source code 

Return whether given string is a valid md5 hex digest.

Parameters:
  • string (string) - string to test
Returns: bool
True, if valid

elf_class_strtoint(elf_class_str)

source code 

Convert an ELF class metadataum string to its int value.

Parameters:
  • elf_class_str (string) - the ELF class string
Returns: int
ELF class int value

read_elf_class(elf_file)

source code 

Read ELF class metadatum from ELF file.

Parameters:
  • elf_file (string) - path to ELF file
Returns: int
ELF class metadatum value

is_elf_file(elf_file)

source code 

Determine whether given file path points to an ELF file object.

Parameters:
  • elf_file (string) - path to ELF file
Returns: bool
True, if file at path is ELF file

parse_rpath(rpath)

source code 

Parse RPATH metadata stored in repository and return an ordered list of paths.

Parameters:
  • rpath (string) - raw RPATH metadata string
Returns: list
a list of paths

resolve_dynamic_library(library, requiring_executable)

source code 

Resolve given library name (as contained into ELF metadata) to a library path.

Parameters:
  • library (string) - library name (as contained into ELF metadata)
  • requiring_executable (string) - path to ELF object that contains the given library name
Returns: string
resolved library path

read_elf_dynamic_libraries(elf_file)

source code 

Extract NEEDED metadatum from ELF file at path.

Parameters:
  • elf_file (string) - path to ELF file
Returns: set
list (set) of strings in NEEDED metadatum

read_elf_metadata(elf_file)

source code 

Extract soname, elf class, runpath and NEEDED metadata from ELF file.

Parameters:
  • elf_file (string) - path to ELF file
Returns: dict or None
dict with "soname", "class", "runpath" and "needed" keys. None if no metadata is found.

read_elf_real_dynamic_libraries(elf_file)

source code 

This function is similar to read_elf_dynamic_libraries but uses ldd to retrieve a list of "real" .so library dependencies used by the ELF file. This is useful to ensure that there are no .so libraries missing in the dependencies, because ldd expands and resolves the .so dependency graph. This is anyway dangerous because the output returned by ldd is somehow environment-dependent, so make sure this function is only used for informative purposes, and not for adding real dependencies to a package.

Parameters:
  • elf_file (string) - path to ELF file
Returns: set
list (set) of strings in NEEDED metadatum
Raises:

read_elf_broken_symbols(elf_file)

source code 

Extract broken symbols from ELF file.

Parameters:
  • elf_file (string) - path to ELF file
Returns: set
list of broken symbols in ELF file.

read_elf_linker_paths(elf_file)

source code 

Extract built-in linker paths (RUNPATH and RPATH) from ELF file.

Parameters:
  • elf_file (string) - path to ELF file
Returns: list
list of extracted built-in linker paths.

xml_from_dict_extended(dictionary)

source code 

Serialize a simple dict object into an XML string.

Parameters:
  • dictionary (dict) - dict object
Returns: string
XML string representing the dict object

dict_from_xml_extended(xml_string)

source code 

Deserialize an XML string representing a dict object back into a dict object. WARNING: eval() is used for non-string, non-bool types.

Parameters:
  • xml_string (string) - string to deserialize
Returns: dict
reconstructed dict object

xml_from_dict(dictionary)

source code 

Serialize a dict object into a "simple" XML string. This method is faster and safer than xml_from_dict_extended but it doesn't support dict values and keys different from strings.

Parameters:
  • dictionary (dict) - dictionary object
Returns: string
serialized XML string

dict_from_xml(xml_string)

source code 

Deserialize an XML string representing a dict (created by xml_from_dict) back into a dict object. This method is faster and safer than dict_from_xml_extended but it doesn't support dict values and keys different from strings.

Parameters:
  • xml_string (string) - XML string to deserialize
Returns: dict
deserialized dict object

collect_linker_paths()

source code 

Collect dynamic linker paths set into /etc/ld.so.conf. This function is ROOT safe.

Returns: tuple
list of dynamic linker paths set

collect_paths()

source code 

Return env var PATH value split using ":" as separator.

Returns: list
list of PATHs

create_package_dirpath(branch, nonfree=False, restricted=False)

source code 

Create Entropy package relative directory path used for building EntropyRepository "download" metadatum and for handling package file life by Entropy Server.

Parameters:
  • branch (string) - Entropy branch id
  • nonfree (bool) - if package belongs to free or nonfree dir
Returns: string
complete relative path

recursive_directory_relative_listing(empty_list, base_directory, _nested=False)

source code 

Takes an array(list) and appends all files from dir down the directory tree. Returns nothing. list is modified.

flatten(mylist)

source code 

Recursively traverse nested lists and return a single list containing all non-list elements that are found.

Parameters:
  • mylist (List) - A list containing nested lists and non-list elements.
Returns: List
A single list containing only non-list elements.

total_memory()

source code 

Return the amount of total system memory in megabytes.

Returns: int
the total system memory available