Skip to content

Figshare download links #338

Description

@MiraAhmedovic

In both Hinojosa-2024-Emotions and Coso-2023-Emotions, the figshare download link needed a lot of tweaking since the website was redirecting any "normal" requests and only allows an automated download if we provide a broswer identity. If these aren't provided, an empy file is downloaded and the mapping command fails.
It might be necesarry to check any other figshare download links and their functionality and, if necesarry, replace the download command.
The link that can be copied from the website needs to be changed a litte. If it looks like this: https://figshare.com/ndownloader/files/36434421 it has to be changed to look like this: https://ndownloader.figshare.com/files/36434421
Then, the download command can be changed as follows:

import ssl
from urllib.request import urlopen, Request
from pathlib import Path
def download(dataset):
    URL = "https://ndownloader.figshare.com/CHANGED-LINK-HERE"
    FILE_NAME = "NAME-OF-FILE"

    save_dir = Path(__file__).resolve().parent / "raw"
    save_dir.mkdir(parents=True, exist_ok=True)

    ctx = ssl.create_default_context(ssl.Purpose.SERVER_AUTH)
    ctx.options |= 0x4 

    req = Request(URL, headers={"User-Agent": "Mozilla/5.0"})

    with urlopen(req, context=ctx) as resp:
        content = resp.read()

    file_path = save_dir / FILE_NAME

    with open(file_path, "wb") as f:
        f.write(content)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions