radar_api package

Subpackages

• radar_api.utils package
  • Submodules
  • radar_api.utils.list module
    • flatten_list()
  • radar_api.utils.timing module
    • print_elapsed_time()
  • radar_api.utils.xradar module
    • get_nexrad_datatree_from_pyart()
  • radar_api.utils.yaml module
    • read_yaml()
    • write_yaml()
  • Module contents

Submodules

radar_api.checks module

This module provides functions to check RADAR-API arguments.

radar_api.checks.check_base_dir(base_dir)

Check base_dir validity.

radar_api.checks.check_date(date)

Ensure the returned object is a datetime.date object.

radar_api.checks.check_download_protocol(protocol)

Check protocol validity for download.

radar_api.checks.check_network(network)

Check radar network validity.

radar_api.checks.check_protocol(protocol)

Check protocol validity.

radar_api.checks.check_radar(radar, network)

Check radar name validity.

radar_api.checks.check_start_end_time(start_time, end_time)

Check start_time and end_time validity.

radar_api.checks.check_time(time)

Check time validity.

It returns a datetime.datetime object to seconds precision.

Parameters:

time (datetime.datetime, datetime.date, numpy.datetime64 or str) – Time object. Accepted types: datetime.datetime, datetime.date, numpy.datetime64 or str. If string type, it expects the isoformat YYYY-MM-DD hh:mm:ss.

Returns:

time

Return type:

datetime.datetime

radar_api.checks.get_current_utc_time()

Return current UTC time.

radar_api.configs module

RADAR-API configurations settings.

radar_api.configs.define_configs(base_dir: str | None = None)

Defines the RADAR-API configuration file with the given credentials and base directory.

Parameters:

base_dir (str) – The base directory where radar data are stored.

Notes

This function writes a YAML file to the user’s home directory at ~/.config_radar_api.yaml with the given RADAR-API credentials and base directory. The configuration file can be used for authentication when making RADAR-API requests.

radar_api.configs.get_base_dir(base_dir=None)

Return the RADAR-API base directory.

radar_api.configs.read_configs() → dict[str, str]

Reads the RADAR-API configuration file and returns a dictionary with the configuration settings.

Returns:

A dictionary containing the configuration settings for the RADAR-API.

Return type:

dict

Raises:

ValueError – If the configuration file has not been defined yet. Use radar_api.define_configs() to specify the configuration file path and settings.

Notes

This function reads the YAML configuration file located at ~/.config_radar_api.yaml, which should contain the RADAR-API credentials and base directory specified by radar_api.define_configs().

radar_api.download module

Define download functions.

radar_api.download.create_local_directories(fpaths, exist_ok=True)

Create recursively local directories for the provided filepaths.

radar_api.download.define_local_filepath(filename, network, radar, base_dir=None)

Define filepath where to save file locally on disk.

radar_api.download.get_end_of_day(time)

Get datetime end of the day.

radar_api.download.get_list_daily_time_blocks(start_time, end_time)

Return a list of (start_time, end_time) tuple of daily length.

radar_api.download.get_start_of_day(time)

Get datetime start of the day.

radar_api.download.remove_corrupted_files(local_fpaths, bucket_fpaths, fs, return_corrupted_fpaths=True)

Check and remove files from local disk which are corrupted.

Corruption is evaluated by comparing the size of data on local storage against size of data located in the cloud bucket.

Parameters:

• local_fpaths (list) – List of filepaths on local storage.

• bucket_fpaths (list) – List of filepaths on cloud bucket.

• fs (fsspec.FileSystem) – fsspec filesystem instance. It must be cohrenet with the cloud bucket address of bucket_fpaths.

• return_corrupted_fpaths (bool, optional) – If True, it returns the list of corrupted files. If False, it returns the list of valid files. The default is True.

Returns:

(list_local_filepaths, list_bucket_filepaths)

Return type:

tuple

radar_api.filter module

This module provides files filtering functions.

radar_api.filter.filter_file(fpath, network, start_time, end_time)

Utility function to select a file is matching the specified time periods.

radar_api.filter.filter_files(fpaths, network, start_time=None, end_time=None)

Utility function to select filepaths between time periods.

radar_api.filter.is_file_within_time(start_time, end_time, file_start_time, file_end_time)

Check if a file is within start_time and end_time.

radar_api.info module

This module provides tools to extract information from radar filenames.

radar_api.info.check_groups(groups)

Check groups validity.

radar_api.info.get_end_time_from_filepaths(filepaths, network, ignore_errors=False)

Infer files end_time from filenames.

radar_api.info.get_info_from_filename(filename, network, ignore_errors=False)

Retrieve file information dictionary from filename.

radar_api.info.get_info_from_filepath(filepath, network, ignore_errors=False)

Retrieve file information dictionary from filepath.

radar_api.info.get_key_from_filepath(filepath, key, network, ignore_errors=False)

Extract specific key information from a list of filepaths.

radar_api.info.get_key_from_filepaths(filepaths, key, network, ignore_errors=False)

Extract specific key information from a list of filepaths.

radar_api.info.get_season(time)

Get season from datetime.datetime or datetime.date object.

radar_api.info.get_start_end_time_from_filepaths(filepaths, network, ignore_errors=False)

Infer files start_time and end_time from filenames.

radar_api.info.get_start_time_from_filepaths(filepaths, network, ignore_errors=False)

Infer files start_time from filenames.

radar_api.info.get_time_component(time, component)

Get time component from datetime.datetime object.

radar_api.info.get_version_from_filepath(filepath, network, integer=True)

Infer file version from filenames.

radar_api.info.get_version_from_filepaths(filepaths, network, integer=True)

Infer files version from filenames.

radar_api.info.group_filepaths(filepaths, network, groups=None)

Group filepaths in a dictionary if groups are specified.

Parameters:

• filepaths (list) – List of filepaths.

• groups (list or str) – The group keys by which to group the filepaths. Valid group keys are start_time, end_time, version, volume_identifier, radar_acronym, extension, year, month, day, doy, dow, hour, minute, second, month_name, quarter, season. The time components are extracted from start_time ! If groups is None returns the input filepaths list. The default is None.

Returns:

Either a dictionary of format {<group_value>: <list_filepaths>}. or the original input filepaths (if groups=None)

Return type:

dict or list

radar_api.info.parse_filename(filename, network)

Try to parse the filename based on the radar network.

radar_api.io module

Define filesystems, buckets, connection types and directory structures.

radar_api.io.available_networks()

Get list of available networks.

radar_api.io.available_radars(network=None, start_time=None, end_time=None)

Get list of available radars.

radar_api.io.get_bucket_prefix(protocol)

Get protocol prefix.

radar_api.io.get_database()

Retrieve the RADAR-API database.

radar_api.io.get_directory_pattern(protocol, network)

Get directory pattern.

radar_api.io.get_filesystem(protocol, fs_args=None)

Define fsspec filesystem.

protocolstr

String specifying the cloud bucket storage from which to retrieve the data. It must be specified if not searching data on local storage. Use goes_api.available_protocols() to retrieve available protocols.

fs_argsdict, optional

Dictionary specifying optional settings to initiate the fsspec.filesystem. The default is an empty dictionary. Anonymous connection is set by default.

radar_api.io.get_network_config_filepath(network)

Get filepath of the network configuration file.

radar_api.io.get_network_config_path()

Get directory path with the network configuration files.

radar_api.io.get_network_database(network)

Retrieve the radar network database.

radar_api.io.get_network_filename_patterns(network)

Get radar filenames patterns.

radar_api.io.get_network_info(network)

Get network information.

radar_api.io.get_network_radars_config_path(network)

Get directory path with the radar configuration files of a given network.

radar_api.io.get_radar_config_filepath(network, radar)

Get filepath of the radar configuration file.

radar_api.io.get_radar_end_time(network, radar)

Get last timestep with radar data.

radar_api.io.get_radar_info(network, radar)

Get radar information.

radar_api.io.get_radar_location(network, radar)

Get radar location.

radar_api.io.get_radar_start_time(network, radar)

Get first timestep with radar data.

radar_api.io.get_radar_time_coverage(network, radar)

Get first and last timestep with radar data.

radar_api.io.is_radar_available(network, radar, start_time=None, end_time=None)

Check if a radar was existing within the specified time period.

If start_time and end_time are None, does not perform any check.

Parameters:

• start_time (datetime.datetime, datetime.date, numpy.datetime64 or str) – Start time. Accepted types: datetime.datetime, datetime.date, numpy.datetime64 or str If string type, it expects the isoformat YYYY-MM-DD hh:mm:ss.

• end_time (datetime.datetime, datetime.date, numpy.datetime64 or str) – Start time. Accepted types: datetime.datetime, datetime.date, numpy.datetime64 or str If string type, it expects the isoformat YYYY-MM-DD hh:mm:ss. If None, assume current UTC time.

radar_api.readers module

This module defines file readers.

radar_api.readers.check_software_availability(software, conda_package)

A decorator to ensure that a software package is installed.

Parameters:

• software (str) – The package name as recognized by Python’s import system.

• conda_package (str) – The package name as recognized by conda-forge.

radar_api.readers.get_pyart_reader(network)

Return the pyart reader.

radar_api.readers.get_simplecache_file(filepath)

Simple cache a s3 file.

radar_api.readers.get_xradar_datatree_reader(network)

Return the xradar datatree reader.

radar_api.readers.get_xradar_engine(network)

Return the xradar engine.

radar_api.readers.open_dataset(filepath, network, sweep, **kwargs)

Open a file into an xarray Dataset object using xradar.

radar_api.readers.open_datatree(filepath, network, **kwargs)

Open a file into an xarray DataTree object using xradar.

radar_api.readers.open_pyart(filepath, network, **kwargs)

Open a file into a pyart object.

radar_api.search module

This module provides functions for searching files on local disk and cloud buckets.

radar_api.search.find_files(radar, network, start_time, end_time, base_dir=None, protocol='s3', fs_args={}, verbose=False)

Retrieve files from local or cloud bucket storage.

Parameters:

• base_dir (str, optional) – This argument must be specified only if searching files on the local storage when protocol=”file”. It represents the path to the local directory where to search for radar data. If protocol=”file” and base_dir is None, base_dir is retrieved from the RADAR-API config file. The default is None.

• protocol (str (optional)) – String specifying the location where to search for the data. If protocol=”file”, it searches on local storage (indicated by base_dir). Otherwise, protocol refers to a specific cloud bucket storage. Use radar_api.available_protocols() to check the available protocols. The default is “s3”.

• fs_args (dict, optional) – Dictionary specifying optional settings to initiate the fsspec.filesystem. The default is an empty dictionary. Anonymous connection is set by default.

• radar (str) – The name of the radar. Use radar_api.available_radars() to retrieve the available satellites.

• network (str) – The name of the radar network. See radar_api.available_network() for available radar networks.

• start_time (datetime.datetime) – The start (inclusive) time of the interval period for retrieving the filepaths.

• end_time (datetime.datetime) – The end (exclusive) time of the interval period for retrieving the filepaths.

• verbose (bool, optional) – If True, it print some information concerning the file search. The default is False.

radar_api.search.get_directories_paths(start_time, end_time, network, radar, protocol, base_dir)

Returns a list of the directory paths to scan.

radar_api.search.get_list_timesteps(start_time, end_time, freq)

Return the list of timesteps (directory) to scan.

radar_api.search.get_pattern_shortest_time_component(directory_pattern)

Return the shortest time frequency component present in the pattern.

Module contents

RADAR-API Package.

radar_api.available_networks()

Get list of available networks.

radar_api.available_radars(network=None, start_time=None, end_time=None)

Get list of available radars.

radar_api.define_configs(base_dir: str | None = None)

Defines the RADAR-API configuration file with the given credentials and base directory.

Parameters:

base_dir (str) – The base directory where radar data are stored.

Notes

This function writes a YAML file to the user’s home directory at ~/.config_radar_api.yaml with the given RADAR-API credentials and base directory. The configuration file can be used for authentication when making RADAR-API requests.

radar_api.download_files(*args, **kwargs)

radar_api.find_files(radar, network, start_time, end_time, base_dir=None, protocol='s3', fs_args={}, verbose=False)

Retrieve files from local or cloud bucket storage.

Parameters:

• base_dir (str, optional) – This argument must be specified only if searching files on the local storage when protocol=”file”. It represents the path to the local directory where to search for radar data. If protocol=”file” and base_dir is None, base_dir is retrieved from the RADAR-API config file. The default is None.

• protocol (str (optional)) – String specifying the location where to search for the data. If protocol=”file”, it searches on local storage (indicated by base_dir). Otherwise, protocol refers to a specific cloud bucket storage. Use radar_api.available_protocols() to check the available protocols. The default is “s3”.

• fs_args (dict, optional) – Dictionary specifying optional settings to initiate the fsspec.filesystem. The default is an empty dictionary. Anonymous connection is set by default.

• radar (str) – The name of the radar. Use radar_api.available_radars() to retrieve the available satellites.

• network (str) – The name of the radar network. See radar_api.available_network() for available radar networks.

• start_time (datetime.datetime) – The start (inclusive) time of the interval period for retrieving the filepaths.

• end_time (datetime.datetime) – The end (exclusive) time of the interval period for retrieving the filepaths.

• verbose (bool, optional) – If True, it print some information concerning the file search. The default is False.

radar_api.group_filepaths(filepaths, network, groups=None)

Group filepaths in a dictionary if groups are specified.

Parameters:

• filepaths (list) – List of filepaths.

• groups (list or str) – The group keys by which to group the filepaths. Valid group keys are start_time, end_time, version, volume_identifier, radar_acronym, extension, year, month, day, doy, dow, hour, minute, second, month_name, quarter, season. The time components are extracted from start_time ! If groups is None returns the input filepaths list. The default is None.

Returns:

Either a dictionary of format {<group_value>: <list_filepaths>}. or the original input filepaths (if groups=None)

Return type:

dict or list

radar_api.open_dataset(filepath, network, sweep, **kwargs)

Open a file into an xarray Dataset object using xradar.

radar_api.open_datatree(filepath, network, **kwargs)

Open a file into an xarray DataTree object using xradar.

radar_api.open_pyart(filepath, network, **kwargs)

Open a file into a pyart object.

radar_api.read_configs() → dict[str, str]

Reads the RADAR-API configuration file and returns a dictionary with the configuration settings.

Returns:

A dictionary containing the configuration settings for the RADAR-API.

Return type:

dict

Raises:

ValueError – If the configuration file has not been defined yet. Use radar_api.define_configs() to specify the configuration file path and settings.

Notes

This function reads the YAML configuration file located at ~/.config_radar_api.yaml, which should contain the RADAR-API credentials and base directory specified by radar_api.define_configs().