Miscellaneous operations.

General use

confirmed([prompt, confirmation_required, resp])

Type to confirm whether to proceed or not.

get_obj_attr(obj[, col_names, as_dataframe])

Get main attributes of an object.


Convert a string to its intrinsic data type.


Convert standard GPS time to UTC time.

parse_size(size[, binary, precision])

Parse size from / into readable format of bytes.

get_number_of_chunks(file_or_obj[, ...])

Get total number of chunks of a data file, given a minimum limit of chunk size.

find_executable(app_name[, possibilities])

Get pathname of an executable file for a specified application.

hash_password(password[, salt, salt_size, ...])

Hash a password using hashlib.pbkdf2_hmac.

verify_password(password, salt, key[, ...])

Verify a password given salt and key.

Basic data manipulation



Get every pair (current, next).

split_list_by_size(lst, sub_len)

Split a list into (evenly sized) sub-lists.

split_list(lst, num_of_sub)

Split a list into a number of equally-sized sub-lists.

split_iterable(iterable, chunk_size)

Split a list into (evenly sized) chunks.

update_dict(dictionary, updates[, inplace])

Update a (nested) dictionary or similar mapping.

update_dict_keys(dictionary[, replacements])

Update keys in a (nested) dictionary.

get_dict_values(key, dictionary)

Get all values in a (nested) dictionary for a given key.

remove_dict_keys(dictionary, *keys)

Remove multiple keys from a dictionary.

compare_dicts(dict1, dict2)

Compare the difference between two dictionaries.


Merge multiple dictionaries.

Tabular data

detect_nan_for_str_column(data_frame[, ...])

Detect if a str type column contains NaN when reading csv files.


Create a rotation matrix (counterclockwise).

dict_to_dataframe(input_dict[, k, v])

Convert a dictionary to a data frame.

parse_csr_matrix(path_to_csr[, verbose])

Load in a compressed sparse row (CSR) or compressed row storage (CRS).

swap_cols(array, c1, c2[, as_list])

Swap positions of two columns in an array.

swap_rows(array, r1, r2[, as_list])

Swap positions of two rows in an array.

np_shift(array, step[, fill_value])

Shift an array by desired number of rows.

Basic computation

get_extreme_outlier_bounds(num_dat[, k])

Get upper and lower bounds for extreme outliers.


Calculate interquartile range.

find_closest_date(date, lookup_dates[, ...])

Find the closest date of a given one from a list of dates.

Graph plotting

cmap_discretisation(cmap, n_colours)

Create a discrete colour ramp.

colour_bar_index(cmap, n_colours[, labels])

Create a colour bar.

Web data extraction


Check whether the current machine can connect to the Internet.

is_url(url[, partially])

Check whether url is a valid URL.


Check whether the current machine can connect to a given URL.

is_downloadable(url[, request_field])

Check whether a URL leads to a web page where there is downloadable contents.

init_requests_session(url[, max_retries, ...])

Instantiate a requests session.

load_user_agent_strings([shuffled, ...])

Load user-agent strings of popular browsers.


Get a random user-agent string of a certain browser.


Make a fake HTTP headers for requests.get.

download_file_from_url(url, path_to_file[, ...])

Download an object available at a valid URL.