Package entropy :: Module fetchers :: Class MultipleUrlFetcher

Class MultipleUrlFetcher

source code


Instance Methods
 
__init__(self, url_path_list, checksum=True, show_speed=True, resume=True, abort_check_func=None, disallow_redirect=False, url_fetcher_class=None, timeout=None, download_context_func=None, pre_download_hook=None, post_download_hook=None, http_basic_user=None, http_basic_pwd=None, https_validate_cert=True)
x.__init__(...) initializes x; see help(type(x)) for signature
source code
 
download(self)
Start downloading URL given at construction time.
source code
int
get_transfer_rate(self)
Return transfer rate, in kb/sec.
source code
float
get_average(self)
Get current download percentage.
source code
int
get_seconds_remaining(self)
Return remaining seconds to download completion.
source code
 
handle_statistics(self, th_id, downloaded_size, total_size, average, old_average, update_step, show_speed, data_transfer, time_remaining, time_remaining_secs)
Reimplement this callback to gather information about data currently downloaded.
source code
 
update(self)
Main fetch progress callback.
source code

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __repr__, __setattr__, __sizeof__, __str__, __subclasshook__

Class Methods

Inherited from output.TextInterface: ask_question, edit_file, input_box, output, set_title

Static Methods
bool
supports_differential_download(url)
Return whether the current protocol handler for given URL supports differential download.
source code
Class Variables

Inherited from output.TextInterface: OUTPUT_LOCK

Properties

Inherited from object: __class__

Method Details

__init__(self, url_path_list, checksum=True, show_speed=True, resume=True, abort_check_func=None, disallow_redirect=False, url_fetcher_class=None, timeout=None, download_context_func=None, pre_download_hook=None, post_download_hook=None, http_basic_user=None, http_basic_pwd=None, https_validate_cert=True)
(Constructor)

source code 

x.__init__(...) initializes x; see help(type(x)) for signature

Parameters:
  • url_path_list (list) - list of tuples composed by url and path to save, for eg. [(url,path_to_save,),...]
  • checksum (bool) - return md5 hash instead of status code
  • show_speed (bool) - show download speed
  • resume (bool) - enable resume support
  • abort_check_func (callable) - callback used to stop download, it has to raise an exception that has to be caught by provider application. This exception will be considered an "abort" request.
  • disallow_redirect (bool) - disallow automatic HTTP redirects
  • thread_stop_func (callable) - callback used to stop download, it has to raise an exception that has to be caught by provider application. This exception will be considered a "stop" request.
  • url_fetcher_class (subclass of UrlFetcher) - UrlFetcher based class to use
  • timeout (int) - custom request timeout value (in seconds), if None the value is read from Entropy configuration files.
  • download_context_func (callable) - if not None, it must be a function exposing a context manager and taking a path (the download path) as argument. This can be used to implement locking on files to be downloaded.
  • pre_download_hook (callable) - hook called before starting the download process, inside the download_context_func context. This can be used to verify if the download is actually needed or just return. If the returned value is not None, the download method will return that value. The function takes a path (the download path) and the download id as arguments.
  • post_download_hook (callable) - hook called after the download is complete, inside the download_context_func context. This can be used to verify the integrity of the downloaded data. The function takes a path (the download path) and the download status and the download id as arguments.
Overrides: object.__init__

supports_differential_download(url)
Static Method

source code 

Return whether the current protocol handler for given URL supports differential download. It's up to UrlFetcher users to implement the logic to provide a previously downloaded file to "path_to_save" location (argument passed to UrlFetcher constructor).

Parameters:
  • url (string) - download URL
Returns: bool
differential download support status

download(self)

source code 
Start downloading URL given at construction time.

@return: dict containing UrlFetcher.get_id() as key
    and download status as value, which can be either one of:
    UrlFetcher.GENERIC_FETCH_ERROR means error.
    UrlFetcher.TIMEOUT_FETCH_ERROR means timeout error.
    UrlFetcher.GENERIC_FETCH_WARN means warning,
        downloaded fine but unable to calculate the md5 hash.
@rtype: dict

get_transfer_rate(self)

source code 

Return transfer rate, in kb/sec.

Returns: int
transfer rate

get_average(self)

source code 

Get current download percentage.

Returns: float
download percentage

get_seconds_remaining(self)

source code 

Return remaining seconds to download completion.

Returns: int
remaining download seconds

handle_statistics(self, th_id, downloaded_size, total_size, average, old_average, update_step, show_speed, data_transfer, time_remaining, time_remaining_secs)

source code 

Reimplement this callback to gather information about data currently downloaded.

Parameters:
  • th_id (int) - instance identifier
  • downloaded_size (int) - size downloaded up to now, in bytes
  • total_size (int) - total download size, in bytes
  • average (float) - percentage of file downloaded up to now
  • old_average - old percentage of file downloaded
  • update_step (int) - currently configured update average delta
  • show_speed (bool) - if download speed should be shown for given instance
  • data_transfer (int) - current data transfer, in kb/sec
  • time_remaining (string) - currently hypothesized remaining download time, in string format (showing hours, minutes, seconds).
  • time_remaining_secs (int) - currently hypothesized remaining download time, in seconds.

update(self)

source code 

Main fetch progress callback. You can reimplement this to refresh your output devices.