ops

Miscellaneous operations.

General use

confirmed([prompt, confirmation_required, resp])

Type to confirm whether to proceed or not.

get_obj_attr(obj[, col_names])

Get main attributes of an object.

eval_dtype(str_val)

Convert a string to its intrinsic data type.

gps_to_utc(gps_time)

Convert standard GPS time to UTC time.

parse_size(size[, binary, precision])

Parse size from / into readable format of bytes.

get_number_of_chunks(file_or_obj[, ...])

Get total number of chunks of a data file, given a minimum limit of chunk size.

Basic data manipulation

Iterable

loop_in_pairs(iterable)

A function to iterate a list as pair (current, next).

split_list_by_size(lst, sub_len)

Split a list into (evenly sized) sub-lists.

split_list(lst, num_of_sub)

Split a list into a number of equally-sized sub-lists.

split_iterable(iterable, chunk_size)

Split a list into (evenly sized) chunks.

update_dict(dictionary, updates[, inplace])

Update a (nested) dictionary or similar mapping.

update_dict_keys(dictionary[, replacements])

Update keys in a (nested) dictionary.

get_dict_values(key, dictionary)

Get all values in a (nested) dictionary for a given key.

remove_dict_keys(dictionary, *keys)

Remove multiple keys from a dictionary.

merge_dicts(*dicts)

Merge multiple dictionaries.

Tabular data

detect_nan_for_str_column(data_frame[, ...])

Detect if a str type column contains NaN when reading csv files.

create_rotation_matrix(theta)

Create a rotation matrix (counterclockwise).

dict_to_dataframe(input_dict[, k, v])

Convert a dictionary to a data frame.

parse_csr_matrix(path_to_csr[, verbose])

Load in a compressed sparse row (CSR) or compressed row storage (CRS).

swap_cols(array, c1, c2[, as_list])

Swap positions of two columns in an array.

swap_rows(array, r1, r2[, as_list])

Swap positions of two rows in an array.

Basic computation

get_extreme_outlier_bounds(num_dat[, k])

Get upper and lower bounds for extreme outliers.

interquartile_range(num_dat)

Calculate interquartile range.

find_closest_date(date, lookup_dates[, ...])

Find the closest date of a given one from a list of dates.

Graph plotting

cmap_discretisation(cmap, n_colours)

Create a discrete colour ramp.

colour_bar_index(cmap, n_colours[, labels])

Create a colour bar.

Web data extraction

is_network_connected()

Check whether the current machine can connect to the Internet.

is_url(url[, partially])

Check whether url is a valid URL.

is_url_connectable(url)

Check whether the current machine can connect to a given URL.

is_downloadable(url[, request_field])

Check whether a URL leads to a web page where there is downloadable contents.

init_requests_session(url[, max_retries, ...])

Instantiate a requests session.

get_user_agent_strings([shuffled, ...])

Get user-agent strings for some popular browsers.

get_user_agent_string([fancy])

Get a random user-agent string of a certain browser.

fake_requests_headers([randomized])

Make a fake HTTP headers for requests.get.

download_file_from_url(url, path_to_file[, ...])

Download an object available at a valid URL.