Skip to content

Add create_pandas_dataframe option to load() #33

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
atalha1 wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Add create_pandas_dataframe option to load() #33

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions simfin/load.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@
# Load function.

def load(dataset, variant=None, market=None, start_date=None, end_date=None,
parse_dates=None, index=None, refresh_days=30):
parse_dates=None, index=None, refresh_days=30,
create_pandas_dataframe=True):
"""
Load the dataset from local disk and return it as a Pandas DataFrame.
If the dataset does not exist on local disk, or if it is too old, then it
Expand Down Expand Up @@ -120,8 +121,16 @@ def load(dataset, variant=None, market=None, start_date=None, end_date=None,
A value of refresh_days == 0 means the data is downloaded regardless
of the age of the data-files saved on disk.

:param create_pandas_dataframe:
Boolean (default True). If True, load the CSV-file into a Pandas
DataFrame and return it as usual. If False, skip that step and just
return the path to the CSV-file on disk. Useful if you only want to
download the data and use something other than Pandas to read it,
while still getting the caching / refresh-days handling for free.

:return:
Pandas DataFrame with the data.
Pandas DataFrame with the data, or a string with the path to the
CSV-file on disk if create_pandas_dataframe=False.
"""

assert dataset is not None
Expand All @@ -148,6 +157,12 @@ def load(dataset, variant=None, market=None, start_date=None, end_date=None,
print('\n- Applying filter ... ', end='')
path = _filtered_file(path, start_date, end_date=end_date)

# If the caller doesn't want a DataFrame, just hand back the path.
# The file is already downloaded / cached / filtered at this point.
if not create_pandas_dataframe:
print('Done!')
return path

# Load dataset into Pandas DataFrame.
# The dataset date columns use ISO-style strings (YYYY-MM-DD), so
# Pandas can parse them directly without a custom callback. This keeps
Expand Down