api-entities/works/work-object/README.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "[![GitHub Repository](https://img.shields.io/badge/GitHub-Repository-181717?style=for-the-badge&logo=GitHub&link=https://github.com/Mearman/openalex-docs/tree/main)](https://github.com/Mearman/openalex-docs/tree/main)[![Open in GitHub](https://img.shields.io/badge/Open%20in-GitHub-181717?style=for-the-badge&logo=github&link=https://github.com/Mearman/openalex-docs/blob/main/api-entities/works/work-object/README.ipynb)](https://github.com/Mearman/openalex-docs/blob/main/api-entities/works/work-object/README.ipynb)[![Open in Colab](https://img.shields.io/badge/Open%20in-Colab-F9AB00?style=for-the-badge&logo=Google%20Colab&link=https://colab.research.google.com/github/Mearman/openalex-docs/blob/main/api-entities/works/work-object/README.ipynb)](https://colab.research.google.com/github/Mearman/openalex-docs/blob/main/api-entities/works/work-object/README.ipynb)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "%pip install --upgrade \"git+https://github.com/Mearman/openalex-python-pydantic-v1.git\"\n",
        "%pip install pandasai"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "import json\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "from openalex_api import Configuration, ApiClient, AutocompleteApi, AuthorsApi, ConceptsApi, FundersApi, InstitutionsApi, PublishersApi, SourcesApi, WorksApi\n",
        "\n",
        "configuration = Configuration(host=\"https://api.openalex.org\")\n",
        "autocomplete_api = AutocompleteApi(ApiClient(configuration))\n",
        "authors_api = AuthorsApi(ApiClient(configuration))\n",
        "concepts_api = ConceptsApi(ApiClient(configuration))\n",
        "funders_api = FundersApi(ApiClient(configuration))\n",
        "institutions_api = InstitutionsApi(ApiClient(configuration))\n",
        "publishers_api = PublishersApi(ApiClient(configuration))\n",
        "sources_api = SourcesApi(ApiClient(configuration))\n",
        "works_api = WorksApi(ApiClient(configuration))\n",
        "\n",
        "from pandasai import SmartDataframe\n",
        "from pandasai.llm import OpenAI"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "# @title  { run: \"auto\", display-mode: \"form\" }\n",
        "openapi_token = \"\" # @param {type:\"string\"}"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Work object\n",
        "\n",
        "There's a lot of useful data inside a work. When you use the API to get a [single work](./../get-a-single-work.ipynb) or [lists of works](./../get-lists-of-works.ipynb), this is what's returned.\n",
        "\n",
        "### `abstract_inverted_index`\n",
        "\n",
        "_Object:_ The abstract of the work, as an [inverted index](https://en.wikipedia.org/wiki/Inverted_index), which encodes information about the abstract's words and their positions within the text. [Like Microsoft Academic Graph](https://web.archive.org/web/20220721160540/https://docs.microsoft.com/en-us/academic-services/graph/resources-faq#what-format-are-paper-abstracts-published-in), OpenAlex doesn't include plaintext abstracts due to legal constraints.\n",
        "\n",
        "```json\n",
        "abstract_inverted_index: {\n",
        "    Despite: [\n",
        "        0\n",
        "    ],\n",
        "    growing: [\n",
        "        1\n",
        "    ],\n",
        "    interest: [\n",
        "        2\n",
        "    ],\n",
        "    in: [\n",
        "        3,\n",
        "        57,\n",
        "        73,\n",
        "        110,\n",
        "        122\n",
        "    ],\n",
        "    Open: [\n",
        "        4,\n",
        "        201\n",
        "    ],\n",
        "    Access: [\n",
        "        5\n",
        "    ],\n",
        "    ...\n",
        "}\n",
        "```\n",
        "\n",
        "#### Abstract inverted index coverage\n",
        "\n",
        "Newer works are more likely to have an abstract inverted index. For example, over 60% of works in 2022 have abstract data, compared to 45% for works older than 2000. Full chart is below:\n",
        "\n",
        "![](<../../../.gitbook/assets/OpenAlex works with abstracts by year (percent) (1).svg>)\n",
        "\n",
        "### `alternate_host_venues` (deprecated)\n",
        "\n",
        "{% hint style=\"danger\" %}\n",
        "The `host_venue` and `alternate_host_venues` properties have been deprecated in favor of [`primary_location`](./README.md#primary_location) and [`locations`](./README.md#locations). The attributes `host_venue` and `alternate_host_venues` are no longer available in the Work object, and trying to access them in filters or group-bys will return an error.\n",
        "{% endhint %}\n",
        "\n",
        "### `authorships`\n",
        "\n",
        "_List:_ List of [`Authorship`](./authorship-object.md) objects, each representing an author and their institution. [Limited to](./../../authors/limitations.ipynb) the first 100 authors to maintain API performance.\n",
        "\n",
        "```json\n",
        "authorships: [\n",
        "    // first authorship object:\n",
        "    {\n",
        "        author_position: \"middle\",\n",
        "        author: {\n",
        "            id: \"https://openalex.org/A5023888391\",\n",
        "            display_name: \"Jason Priem\",\n",
        "            orcid: \"https://orcid.org/0000-0001-6187-6610\"\n",
        "        },\n",
        "        institutions: [\n",
        "            {\n",
        "                id: \"https://openalex.org/I4200000001\",\n",
        "                display_name: \"OurResearch\",\n",
        "                ror: \"https://ror.org/02nr0ka47\",\n",
        "                country_code: \"US\",\n",
        "                type: \"nonprofit\"\n",
        "            }\n",
        "        ]\n",
        "    },\n",
        "    \n",
        "    // more authorship objects go here, omited for space.\n",
        "]\n",
        "```\n",
        "\n",
        "### `apc_list`\n",
        "\n",
        "_Object:_ Information about this work's APC ([article processing charge](https://en.wikipedia.org/wiki/Article_processing_charge)). The object contains:\n",
        "\n",
        "* `value`: _Integer_\n",
        "* `currency`: _String_\n",
        "* `provenance`: _String_ — the source of this data. Currently the only value is “doaj” (DOAJ)\n",
        "* `value_usd`: _Integer_ — the APC converted into USD\n",
        "\n",
        "This value is the APC list price–the price as listed by the journal’s publisher. That’s not always the price _actually_ paid, because publishers may offer various discounts to authors. Unfortunately we don’t always know this discounted price, but when we do you can find it in [`apc_paid`](./README.md#apc_paid).\n",
        "\n",
        "Currently our only source for this data is [DOAJ](https://doaj.org/), and so `doaj` is the only value for `apc_list.provenance`, but we’ll add other sources over time.\n",
        "\n",
        "We currently don’t have information on the list price for hybrid journals (toll-access journals that also provide an open-access option), but we will add this at some point. We do have [`apc_paid`](./README.md#apc_paid) information for hybrid OA works occasionally.\n",
        "\n",
        "You can use this attribute to find works published in [Diamond open access](https://en.wikipedia.org/wiki/Diamond_open_access) journals by looking at works where `apc_list.value` is zero. See [`open_access.oa_status`](./README.md#oa_status) for more info.\n",
        "\n",
        "```json\n",
        "apc_payment: {\n",
        "    value: 3200,\n",
        "    currency: \"USD\",\n",
        "    value_usd: 3200,\n",
        "    provenance: \"doaj\"\n",
        "}\n",
        "```\n",
        "\n",
        "### `apc_paid`\n",
        "\n",
        "_Object:_ Information about the _paid_ APC ([article processing charge](https://en.wikipedia.org/wiki/Article_processing_charge)) for this work. The object contains:\n",
        "\n",
        "* `value`: _Integer_\n",
        "* `currency`: _String_\n",
        "* `provenance`: _String_ — currently either `openapc` or `doaj`, but more will be added; see below for details.\n",
        "* `value_usd`: _Integer_ — the APC converted into USD\n",
        "\n",
        "You can find the _listed_ APC price (when we know it) for a given work using [`apc_list`](./README.md#apc_list). However, authors don’t always pay the listed price; often they get a discounted price from publishers. So it’s useful to know the APC actually paid by authors, as distinct from the list price. This is our effort to provide this.\n",
        "\n",
        "Our best source for the actually paid price is the [OpenAPC](https://openapc.net/) project. Where available, we use that data, and so `apc_paid.provenance` is `openapc`. Where OpenAPC data is unavailable (and unfortunately this is common) we make our best guess by assuming the author paid the APC list price, and apc_paid.provenance will be set to wherever we got the list price from.\n",
        "\n",
        "```json\n",
        "apc_payment: {\n",
        "    value: 2250,\n",
        "    currency: \"EUR\",\n",
        "    value_usd: 2426,\n",
        "    provenance: \"openapc\"\n",
        "}\n",
        "```\n",
        "\n",
        "### `best_oa_location`\n",
        "\n",
        "_Object:_ A [`Location`](./location-object.md) object with the best available open access location for this work.\n",
        "\n",
        "We score open locations to determine which is best using these factors:\n",
        "\n",
        "1. Must have is_oa: true\n",
        "2. type_:_ \"publisher\" is better than \"repository\".\n",
        "3. version: \"publishedVersion\" is better than \"acceptedVersion\", which is better than \"submittedVersion\".\n",
        "4. pdf_url: A location with a direct PDF link is better than one without.\n",
        "5. repository rankings: Some major repositories like PubMed Central and arXiv are ranked above others.\n",
        "\n",
        "```json\n",
        "best_oa_location: {\n",
        "  is_oa: true,\n",
        "  landing_page_url: \"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1398957\",\n",
        "  pdf_url: null,\n",
        "  source: {\n",
        "    id: \"https://openalex.org/S2764455111\",\n",
        "    display_name: \"PubMed Central\",\n",
        "    issn_l: null,\n",
        "    issn: null,\n",
        "    host_organization: \"https://openalex.org/I1299303238\",\n",
        "    type: \"repository\"\n",
        "  },\n",
        "  license: null,\n",
        "  version: \"publishedVersion\"\n",
        "}\n",
        "```\n",
        "\n",
        "### `biblio`\n",
        "\n",
        "_Object:_ Old-timey bibliographic info for this work. This is mostly useful only in citation/reference contexts. These are all strings because sometimes you'll get fun values like \"Spring\" and \"Inside cover.\"\n",
        "\n",
        "* `volume` (_String_)\n",
        "* `issue` (_String_)\n",
        "* `first_page` (_String_)\n",
        "* `last_page` (_String_)\n",
        "\n",
        "```json\n",
        "biblio: {\n",
        "    volume: \"495\",\n",
        "    issue: \"7442\",\n",
        "    first_page: \"437\",\n",
        "    last_page: \"440\"\n",
        "}\n",
        "```\n",
        "\n",
        "### `cited_by_api_url`\n",
        "\n",
        "_String:_ A URL that uses the [`cites`](./../filter-works.md#cites) filter to display a list of works that cite this work. This is a way to expand [`cited_by_count`](./README.md#cited_by_count) into an actual list of works.\n",
        "\n",
        "### `cited_by_count`\n",
        "\n",
        "_Integer:_ The number of citations to this work. These are the times that other works have cited this work: Other works ➞ This work.\n",
        "\n",
        "```json\n",
        "cited_by_count: 382\n",
        "```\n",
        "\n",
        "### `concepts`\n",
        "\n",
        "_List:_ List of dehydrated [`Concept` objects](./../../concepts/concept-object.ipynb).\n",
        "\n",
        "Each `Concept` object in the list also has one additional property:\n",
        "\n",
        "* `score` (_Float_): The strength of the connection between the work and this concept (higher is stronger). This number is produced by AWS Sagemaker, in the last layer of the [machine learning model](https://github.com/ourresearch/openalex-concept-tagging) that assigns concepts.\n",
        "\n",
        "Concepts with a score of at least 0.3 are assigned to the work. However, ancestors of an assigned concept are also added to the work, even if the ancestor scores are below 0.3.\n",
        "\n",
        "{% hint style=\"info\" %}\n",
        "Because ancestor concepts are assigned to works, you may see concepts in works with very low scores, even some zero scores.\n",
        "{% endhint %}\n",
        "\n",
        "```json\n",
        "concepts: [\n",
        "    {\n",
        "        id: \"https://openalex.org/C71924100\",\n",
        "        wikidata: \"https://www.wikidata.org/wiki/Q11190\",\n",
        "        display_name: \"Medicine\",\n",
        "        level: 0,\n",
        "        score: 0.9187037\n",
        "    },\n",
        "    {\n",
        "        id: \"https://openalex.org/C3007834351\",\n",
        "        wikidata: \"https://www.wikidata.org/wiki/Q82069695\",\n",
        "        display_name: \"Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)\",\n",
        "        level: 5,\n",
        "        score: 0.8070164\n",
        "    },\n",
        "    ...\n",
        "    {\n",
        "        id: \"https://openalex.org/C191935318\",\n",
        "        wikidata: \"https://www.wikidata.org/wiki/Q148\",\n",
        "        display_name: \"China\",\n",
        "        level: 2,\n",
        "        score: 0.5948172\n",
        "    },\n",
        "    ...\n",
        "    {\n",
        "        id: \"https://openalex.org/C121608353\",\n",
        "        wikidata: \"https://www.wikidata.org/wiki/Q12078\",\n",
        "        display_name: \"Cancer\",\n",
        "        level: 2,\n",
        "        score: 0.46887803\n",
        "    },\n",
        "    ...\n",
        "    {\n",
        "        id: \"https://openalex.org/C17744445\",\n",
        "        wikidata: \"https://www.wikidata.org/wiki/Q36442\",\n",
        "        display_name: \"Political science\",\n",
        "        level: 0,\n",
        "        score: 0\n",
        "    }\n",
        "]\n",
        "```\n",
        "\n",
        "### `corresponding_author_ids`\n",
        "\n",
        "_List:_ [OpenAlex IDs](./../../../how-to-use-the-api/get-single-entities/README.md#the-openalex-id) of any authors for which [authorships.is_corresponding](./authorship-object.md#is_corresponding) is `true`.\n",
        "\n",
        "```json\n",
        "corresponding_author_ids: [\"https://openalex.org/A5004365451\"]\n",
        "```\n",
        "\n",
        "### `corresponding_institution_ids`\n",
        "\n",
        "_List:_ [OpenAlex IDs](./../../../how-to-use-the-api/get-single-entities/README.md#the-openalex-id) of any institutions found within an `authorship` for which [authorships.is_corresponding](./authorship-object.md#is_corresponding) is `true`.\n",
        "\n",
        "```json\n",
        "corresponding_institution_ids: [\"https://openalex.org/I4210123613\"]\n",
        "```\n",
        "\n",
        "### `countries_distinct_count`\n",
        "\n",
        "_Integer:_ Number of distinct `country_codes` among the [`authorships`](./authorship-object.md#institutions) for this work.\n",
        "\n",
        "```json\n",
        "countries_distinct_count: 4\n",
        "```\n",
        "\n",
        "### `counts_by_year`\n",
        "\n",
        "_List:_ [`Works.cited_by_count`](./README.md#cited_by_count) for each of the last ten years, binned by year. To put it another way: each year, you can see how many times this work was cited.\n",
        "\n",
        "Any citations older than ten years old aren't included. Years with zero citations have been removed so you will need to add those in if you need them.\n",
        "\n",
        "```json\n",
        "counts_by_year: [\n",
        "    {\n",
        "        year: 2022,\n",
        "        cited_by_count: 8\n",
        "    },\n",
        "    {\n",
        "        year: 2021,\n",
        "        cited_by_count: 252\n",
        "    },\n",
        "    ...\n",
        "    {\n",
        "        year: 2012,\n",
        "        cited_by_count: 79\n",
        "    }\n",
        "]\n",
        "```\n",
        "\n",
        "### `created_date`\n",
        "\n",
        "_String:_ The date this `Work` object was created in the OpenAlex dataset, expressed as an [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) date string.\n",
        "\n",
        "```json\n",
        "created_date: \"2017-08-08\"\n",
        "```\n",
        "\n",
        "### `display_name`\n",
        "\n",
        "_String:_ Exactly the same as [`Work.title`](./README.md#title). It's useful for `Work`s to include a `display_name` property, since all the other entities have one.\n",
        "\n",
        "```json\n",
        "display_name: \"The state of OA: a large-scale analysis of the prevalence and impact of Open Access articles\",\n",
        "```\n",
        "\n",
        "### `doi`\n",
        "\n",
        "_String:_ The DOI for the work. This is the [Canonical External ID](./../../../how-to-use-the-api/get-single-entities/README.md#canonical-external-ids) for works.\n",
        "\n",
        "Occasionally, a work has more than one DOI--for example, there might be one DOI for a preprint version hosted on [bioRxiv](https://www.biorxiv.org/), and another DOI for the [published version](./README.md#version). However, this field always has just one DOI, the DOI for the published work.\n",
        "\n",
        "```json\n",
        "doi: \"https://doi.org/10.7717/peerj.4375\"\n",
        "```\n",
        "\n",
        "### `fulltext_origin`\n",
        "\n",
        "_String:_ If a work's full text is searchable in OpenAlex ([`has_fulltext`](./README.md#has_fulltext) is `true`), this tells you how we got the text. This will be one of:\n",
        "\n",
        "* `pdf`: We used [Grobid](https://grobid.readthedocs.io) to get the text from an open-access PDF.\n",
        "* `ngrams`: Full text search is enabled using [N-grams obtained from the Internet Archive](./../get-n-grams.ipynb).\n",
        "\n",
        "This attribute is only available for works with `has_fulltext:true`.\n",
        "\n",
        "```json\n",
        "fulltext_origin: \"pdf\"\n",
        "```\n",
        "\n",
        "### `grants`\n",
        "\n",
        "_List:_ List of grant objects, which include the [`Funder`](./../../funders/README.ipynb) and the award ID, if available. Our grants data comes from Crossref, and is currently fairly limited.\n",
        "\n",
        "```json\n",
        "grants: [\n",
        "    // grant for which we have the grant details:\n",
        "    {\n",
        "        funder: \"https://openalex.org/F4320306076\",\n",
        "        funder_display_name: \"National Science Foundation\",\n",
        "        award_id: \"ABI 1661218\",\n",
        "    },\n",
        "    // grant for which we do not have the details:\n",
        "    {\n",
        "        funder: \"https://openalex.org/F4320306084\",\n",
        "        funder_display_name: \"U.S. Department of Energy\",\n",
        "        award_id: null,\n",
        "    },\n",
        "]\n",
        "```\n",
        "\n",
        "### `has_fulltext`\n",
        "\n",
        "_Boolean:_ Set to `true` if the work's full text is searchable in OpenAlex. This does not necessarily mean that the full text is available to you, dear reader; rather, it means that we have indexed the full text and can use it to help power [searches](./../search-works.ipynb). If you are trying to find the full text for yourself, try looking in [`open_access.oa_url`](./README.md#open_access).\n",
        "\n",
        "We get access to the full text in one of two ways: either using an open-access PDF, or using [N-grams obtained from the Internet Archive](./../get-n-grams.ipynb). You can learn where a work's full text came from at [`fulltext_origin`](./README.md#fulltext_origin).\n",
        "\n",
        "```json\n",
        "has_fulltext: true\n",
        "```\n",
        "\n",
        "### `host_venue` (deprecated)\n",
        "\n",
        "{% hint style=\"danger\" %}\n",
        "The `host_venue` and `alternate_host_venues` properties have been deprecated in favor of [`primary_location`](./README.md#primary_location) and [`locations`](./README.md#locations). The attributes `host_venue` and `alternate_host_venues` are no longer available in the Work object, and trying to access them in filters or group-bys will return an error.\n",
        "{% endhint %}\n",
        "\n",
        "### `id`\n",
        "\n",
        "_String:_ The [OpenAlex ID](./../../../how-to-use-the-api/get-single-entities/README.md#the-openalex-id) for this work.\n",
        "\n",
        "```json\n",
        "id: \"https://openalex.org/W2741809807\"\n",
        "```\n",
        "\n",
        "### `ids`\n",
        "\n",
        "_Object:_ All the external identifiers that we know about for this work. IDs are expressed as URIs whenever possible. Possible ID types:\n",
        "\n",
        "* `doi` (_String:_ The [DOI](https://en.wikipedia.org/wiki/Digital_object_identifier). Same as [`Work.doi`](./README.md#title))\n",
        "* `mag` (_Integer:_ the [Microsoft Academic Graph](https://www.microsoft.com/en-us/research/project/microsoft-academic-graph/) ID)\n",
        "* `openalex` (_String:_ The [OpenAlex ID](./../../../how-to-use-the-api/get-single-entities/README.md#the-openalex-id). Same as [`Work.id`](./README.md#id))\n",
        "* `pmid` (_String:_ The [Pubmed Identifier](https://en.wikipedia.org/wiki/PubMed#PubMed_identifier))\n",
        "* `pmcid` (_String_: the [Pubmed Central identifier](https://www.ncbi.nlm.nih.gov/pmc/about/public-access-info/))\n",
        "\n",
        "{% hint style=\"info\" %}\n",
        "Most works are missing one or more ID types (either because we don't know the ID, or because it was never assigned). Keys for `null` IDs are not displayed.\n",
        "{% endhint %}\n",
        "\n",
        "```json\n",
        "ids: {\n",
        "    openalex: \"https://openalex.org/W2741809807\",\n",
        "    doi: \"https://doi.org/10.7717/peerj.4375\",\n",
        "    mag: 2741809807,\n",
        "    pmid: \"https://pubmed.ncbi.nlm.nih.gov/29456894\"\n",
        "}\n",
        "```\n",
        "\n",
        "### `institutions_distinct_count`\n",
        "\n",
        "_Integer:_ Number of distinct [`institutions`](./../../institutions/README.ipynb) among the [`authorships`](./authorship-object.md#institutions) for this work.\n",
        "\n",
        "```json\n",
        "institutions_distinct_count: 4\n",
        "```\n",
        "\n",
        "### `is_paratext`\n",
        "\n",
        "_Boolean:_ True if we think this work is [paratext](https://en.wikipedia.org/wiki/Paratext).\n",
        "\n",
        "In our context, paratext is stuff that's in scholarly venue (like a journal) but is _about the venue_ rather than a scholarly work properly speaking. Some examples and nonexamples:\n",
        "\n",
        "* **yep it's paratext**: front cover, back cover, table of contents, editorial board listing, issue information, masthead.\n",
        "* **no, not paratext**: research paper, dataset, letters to the editor, figures\n",
        "\n",
        "Turns out there is a lot of paratext in registries like Crossref. That's not a bad thing... but we've found that it's good to have a way to filter it out.\n",
        "\n",
        "We determine `is_paratext` algorithmically using title heuristics.\n",
        "\n",
        "```json\n",
        "is_paratext: false \n",
        "```\n",
        "\n",
        "### `is_retracted`\n",
        "\n",
        "_Boolean:_ True if we know this work has been retracted.\n",
        "\n",
        "We identify works that have been retracted using the public [Retraction Watch database](https://doi.org/10.13003/c23rw1d9), a public resource made possible by a partnership between Crossref and The Center for Scientific Integrity.\n",
        "\n",
        "```json\n",
        "is_retracted: false \n",
        "```\n",
        "\n",
        "### `keywords`\n",
        "\n",
        "_List of objects:_ Keywords extracted from the work's title, along with confidence scores from the extraction model.\n",
        "\n",
        "We use a pretrained model to extract a list of keywords from the work's title. Not all works are assigned keywords. [See the model documentation for more details.](https://github.com/ourresearch/openalex-keywords/tree/main/v1)\n",
        "\n",
        "```json\n",
        "[\n",
        "    {\n",
        "        keyword: \"coastal marine ecosystems\",\n",
        "        score: 0.5626\n",
        "    },\n",
        "    {\n",
        "        keyword: \"kelp detritus\",\n",
        "        score: 0.5403\n",
        "    },\n",
        "    {\n",
        "        keyword: \"secondary production\",\n",
        "        score: 0.3469\n",
        "    }\n",
        "]\n",
        "```\n",
        "\n",
        "### `language`\n",
        "\n",
        "_String:_ The language of the work in [ISO 639-1 format](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes). The language is automatically detected using the information we have about the work. We use the [langdetect](https://pypi.org/project/langdetect/) software library on the words in the work's abstract, or the title if we do not have the abstract. The source code for this procedure is [here.](https://github.com/ourresearch/openalex-guts/blob/54471c6c8e3c59662c4a4d9c37320af7b1667b2b/models/work.py#LL1102C1-L1102C1) Keep in mind that this method is not perfect, and that in some cases the language of the title or abstract could be different from the body of the work.\n",
        "\n",
        "A few things to keep in mind about this:\n",
        "\n",
        "* We don't always assign a language if we do not have enough words available to accurately guess.\n",
        "* We report the language of the metadata, not the full text. For example, if a work is in French, but the title and abstract are in English, we report the language as English.\n",
        "* In some cases, abstracts are in two different languages. Unfortunately, when this happens, what we report will not be accurate.\n",
        "\n",
        "```json\n",
        "language: \"en\"\n",
        "```\n",
        "\n",
        "### `license`\n",
        "\n",
        "_String:_ The license applied to this work at this host. Most toll-access works don't have an explicit license (they're under \"all rights reserved\" copyright), so this field generally has content only if `is_oa` is `true`.\n",
        "\n",
        "```json\n",
        "license: \"cc-by\"\n",
        "```\n",
        "\n",
        "### `locations`\n",
        "\n",
        "_List:_ A list of [`Location`](./location-object.md) objects describing all unique places where this work lives.\n",
        "\n",
        "```json\n",
        "locations: [ \n",
        "  {\n",
        "    is_oa: true,\n",
        "    landing_page_url: \"https://doi.org/10.1073/pnas.17.6.401\",\n",
        "    pdf_url: \"http://www.pnas.org/content/17/6/401.full.pdf\",\n",
        "    source: {\n",
        "      id: \"https://openalex.org/S125754415\",\n",
        "      display_name: \"Proceedings of the National Academy of Sciences of the United States of America\",\n",
        "      issn_l: \"0027-8424\",\n",
        "      issn: [\"1091-6490\", \"0027-8424\"],\n",
        "      host_organization: \"https://openalex.org/P4310320052\",\n",
        "      type: \"journal\"\n",
        "    },\n",
        "    license: null,\n",
        "    version: \"publishedVersion\"\n",
        "  },\n",
        "  {\n",
        "    is_oa: true,\n",
        "    landing_page_url: \"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1076072\",\n",
        "    pdf_url: null,\n",
        "    source: {\n",
        "      id: \"https://openalex.org/S2764455111\",\n",
        "      display_name: \"PubMed Central\",\n",
        "      issn_l: null,\n",
        "      issn: null,\n",
        "      host_organization: \"https://openalex.org/I1299303238\",\n",
        "      type: \"repository\"\n",
        "    },\n",
        "    license: null,\n",
        "    version: \"publishedVersion\"\n",
        "  }\n",
        "]\n",
        "```\n",
        "\n",
        "### `locations_count`\n",
        "\n",
        "_Integer:_ Number of [`locations`](./README.md#locations) for this work.\n",
        "\n",
        "```json\n",
        "locations_count: 3\n",
        "```\n",
        "\n",
        "### `mesh`\n",
        "\n",
        "_List:_ List of [MeSH](https://www.nlm.nih.gov/mesh/meshhome.html) tag objects. Only works found in [PubMed](https://pubmed.ncbi.nlm.nih.gov/) have MeSH tags; for all other works, this is an empty list.\n",
        "\n",
        "```json\n",
        "mesh: [\n",
        "    {\n",
        "        descriptor_ui: \"D017712\",\n",
        "        descriptor_name: \"Peer Review, Research\",\n",
        "        qualifier_ui: \"Q000379\",\n",
        "        qualifier_name: \"methods\",\n",
        "        is_major_topic: false\n",
        "    },\n",
        "    {\n",
        "        descriptor_ui: \"D017712\",\n",
        "        descriptor_name: \"Peer Review, Research\",\n",
        "        qualifier_ui: \"Q000592\",\n",
        "        qualifier_name: \"standards\",\n",
        "        is_major_topic: true\n",
        "    }\n",
        "]\n",
        "```\n",
        "\n",
        "### `ngrams_url`\n",
        "\n",
        "{% hint style=\"info\" %}\n",
        "`ngrams_url` is only displayed in the API and is not included in the [OpenAlex snapshot](./../../../download-all-data/openalex-snapshot.md).\n",
        "{% endhint %}\n",
        "\n",
        "_String:_ It lists groups of words and phrases (n-grams) that make up a work, as obtained from the [Internet Archive](https://archive.org/details/GeneralIndex). See [The Ngram object](./README.md#the-ngram-object) and [Get N-grams](./../get-n-grams.ipynb) for background on n-grams, how we use them, and what this API call returns.\n",
        "\n",
        "```json\n",
        "ngrams_url: \"https://api.openalex.org/works/W2023271753/ngrams\""
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "# @title { run: \"auto\", vertical-output: false }\n",
        "# https://api.openalex.org/works/W2023271753/ngrams\n",
        "id=\"W2023271753\" # @param {type: \"string\"}\n",
        "\n",
        "response = works_api.get_work(\n",
        "\tid=id\n",
        ")\n",
        "\n",
        "df = pd.DataFrame(response).T.rename(columns=lambda x: x[0]).drop(0).set_index('id')\n",
        "display(df)"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "```\n",
        "\n",
        "### `open_access`\n",
        "\n",
        "_Object:_ Information about the access status of this work, as an [`OpenAccess`](./README.md#the-openaccess-object) object.\n",
        "\n",
        "```json\n",
        "open_access: {\n",
        "    is_oa: true,\n",
        "    oa_status: \"gold\",\n",
        "    oa_url: \"https://peerj.com/articles/4375.pdf\",\n",
        "    any_repository_has_fulltext: true\n",
        "},\n",
        "```\n",
        "\n",
        "### `primary_location`\n",
        "\n",
        "_Object:_ A [`Location`](./location-object.md) object with the primary location of this work.\n",
        "\n",
        "The `primary_location` is where you can find the best (closest to the [version of record](https://en.wikipedia.org/wiki/Version_of_record)) copy of this work. For a peer-reviewed journal article, this would be a full text published version, hosted by the publisher at the article's DOI URL.\n",
        "\n",
        "```json\n",
        "primary_location: {\n",
        "  is_oa: true,\n",
        "  landing_page_url: \"https://doi.org/10.1073/pnas.17.6.401\",\n",
        "  pdf_url: \"http://www.pnas.org/content/17/6/401.full.pdf\",\n",
        "  source: {\n",
        "    id: \"https://openalex.org/S125754415\",\n",
        "    display_name: \"Proceedings of the National Academy of Sciences of the United States of America\",\n",
        "    issn_l: \"0027-8424\",\n",
        "    issn: [\"1091-6490\", \"0027-8424\"],\n",
        "    host_organization: \"https://openalex.org/P4310320052\",\n",
        "    type: \"journal\"\n",
        "  },\n",
        "  license: null,\n",
        "  version: \"publishedVersion\"\n",
        "}\n",
        "```\n",
        "\n",
        "### `publication_date`\n",
        "\n",
        "_String:_ The day when this work was published, formatted as an [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) date.\n",
        "\n",
        "Where different publication dates exist, we select the earliest available date of electronic publication.\n",
        "\n",
        "This date applies to the version found at [`Work.url`](./README.md#url). The other versions, found in [`Work.locations`](./README.md#locations), may have been published at different (earlier) dates.\n",
        "\n",
        "```json\n",
        "publication_date: \"2018-02-13\"\n",
        "```\n",
        "\n",
        "### `publication_year`\n",
        "\n",
        "_Integer:_ The year this work was published.\n",
        "\n",
        "This year applies to the version found at [`Work.url`](./README.md#url). The other versions, found in [`Work.locations`](./README.md#locations), may have been published in different (earlier) years.\n",
        "\n",
        "```json\n",
        "publication_year: 2018\n",
        "```\n",
        "\n",
        "### `referenced_works`\n",
        "\n",
        "_List:_ [OpenAlex IDs](./../../../how-to-use-the-api/get-single-entities/README.md#the-openalex-id) for works that this work cites. These are citations that go _from_ this work out _to_ another work: This work ➞ Other works.\n",
        "\n",
        "```json\n",
        "referenced_works: [\n",
        "    \"https://openalex.org/W2753353163\",\n",
        "    \"https://openalex.org/W2785823074\",\n",
        "    \"https://openalex.org/W2511661767\",\n",
        "    \"https://openalex.org/W2115339903\",\n",
        "    \"https://openalex.org/W2031754690\"\n",
        "]\n",
        "```\n",
        "\n",
        "### `related_works`\n",
        "\n",
        "_List:_ [OpenAlex IDs](./../../../how-to-use-the-api/get-single-entities/README.md#the-openalex-id) for works related to this work. Related works are computed algorithmically; the algorithm finds recent papers with the most concepts in common with the current paper.\n",
        "\n",
        "```json\n",
        "related_works: [\n",
        "    \"https://openalex.org/W2753353163\",\n",
        "    \"https://openalex.org/W2785823074\",\n",
        "    \"https://openalex.org/W2511661767\",\n",
        "    \"https://openalex.org/W2115339903\",\n",
        "    \"https://openalex.org/W2031754690\",\n",
        "]\n",
        "```\n",
        "\n",
        "### `sustainable_development_goals`\n",
        "\n",
        "_List:_ List of objects\n",
        "\n",
        "The United Nations' [17 Sustainable Development Goals](https://sdgs.un.org/goals) are a collection of goals at the heart of a global \"shared blueprint for peace and prosperity for people and the planet.\" We use a machine learning model to tag works with their relevance to these goals based on our [OpenAlex SDG Classifier](https://github.com/ourresearch/openalex-sdg-classifier), an mBERT machine learning model developed by the [Aurora Universities Network](https://aurora-universities.eu/sdg-research/). The `score` represents the model's predicted probability of the work's relevance for a particular goal.\n",
        "\n",
        "We display all of the SDGs with a prediction score higher than 0.4.\n",
        "\n",
        "```json\n",
        "sustainable_development_goals: [\n",
        "    {\n",
        "        id: \"https://metadata.un.org/sdg/3\",\n",
        "        display_name: \"Good health and well-being\",\n",
        "        score: \t0.95\n",
        "    }\n",
        "]\n",
        "```\n",
        "\n",
        "### `title`\n",
        "\n",
        "_String:_ The title of this work.\n",
        "\n",
        "This is exactly the same as [`Work.display_name`](./README.md#display_name). We include both attributes with the same information because we want all entities to have a `display_name`, but there's a longstanding tradition of calling this the \"title,\" so we figured you'll be expecting works to have it as a property.\n",
        "\n",
        "```json\n",
        "title: \"The state of OA: a large-scale analysis of the prevalence and impact of Open Access articles\",\n",
        "```\n",
        "\n",
        "### `type`\n",
        "\n",
        "_String:_ The type of the work.\n",
        "\n",
        "You can see all of the different types along with their counts in the OpenAlex API here: [`https://api.openalex.org/works?group_by=type`](https://api.openalex.org/works?group_by=type)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "# @title { run: \"auto\", vertical-output: false }\n",
        "# https://api.openalex.org/works?group_by=type\n",
        "group_by=\"type\" # @param {type: \"string\"}\n",
        "\n",
        "response = works_api.get_works(\n",
        "\tgroup_by=group_by\n",
        ")\n",
        "\n",
        "df = pd.DataFrame(response.group_by)\n",
        "display(df)"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "numeric_df = df.set_index('key')\n",
        "display(numeric_df)"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "try:\n",
        "\tprompt = \"Visualize this data\" # @param {type:\"string\"}\n",
        "\tSmartDataframe(\n",
        "\t\tnumeric_df,\n",
        "\t\tconfig={\"llm\": (OpenAI(api_token=openapi_token))},\n",
        "\t).chat(prompt)\n",
        "except:\n",
        "\tif not openapi_token:\n",
        "\t\tprint(\"Error: openapi_token not set\")\n",
        "\telse:\n",
        "\t\tprint(\"Error when creating SmartDataframe\")"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Most works are type `article`. This includes what was formerly (and currently in [`type_crossref`](./README.md#type_crossref)) labeled as `journal-article`, `proceedings-article`, and `posted-content`. We consider all of these to be `article` type works, and the distinctions between them to be more about where they are published or hosted:\n",
        "\n",
        "* Journal articles will have a [`primary_location.source.type`](./location-object.md#source) of `journal`\n",
        "* Conference proceedings will have a [`primary_location.source.type`](./location-object.md#source) of `conference`\n",
        "* Preprints or \"posted content\" will have a [`primary_location.version`](./location-object.md#version) of `submittedVersion`\n",
        "\n",
        "(Note that distinguishing between journals and conferences is a hard problem, one we often get wrong. We are working on improving this, but we also point out that the two have a lot of overlap in terms of their roles as hosts of research publications.)\n",
        "\n",
        "So, here is how you can filter for only non-preprint articles:\n",
        "\n",
        "* [`https://api.openalex.org/works?filter=type:article,primary_location.version:!submittedVersion`](https://api.openalex.org/works?filter=type:article,primary_location.version:!submittedVersion)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "# @title { run: \"auto\", vertical-output: false }\n",
        "# https://api.openalex.org/works?filter=type:article,primary_location.version:!submittedVersion\n",
        "filter=\"type:article,primary_location.version:!submittedVersion\" # @param {type: \"string\"}\n",
        "\n",
        "response = works_api.get_works(\n",
        "\tfilter=filter\n",
        ")\n",
        "\n",
        "df = pd.DataFrame(response.results)\n",
        "display(df)"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "numeric_df = df[['id', 'display_name'] +\n",
        "\t[col for col in df.columns if df[col].dtype in ['int64', 'float64'] and col != 'relevance_score']]\n",
        "display(numeric_df)"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "try:\n",
        "\tprompt = \"Visualize this data\" # @param {type:\"string\"}\n",
        "\tSmartDataframe(\n",
        "\t\tnumeric_df,\n",
        "\t\tconfig={\"llm\": (OpenAI(api_token=openapi_token))},\n",
        "\t).chat(prompt)\n",
        "except:\n",
        "\tif not openapi_token:\n",
        "\t\tprint(\"Error: openapi_token not set\")\n",
        "\telse:\n",
        "\t\tprint(\"Error when creating SmartDataframe\")"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Works that represent stuff that is _about_ the venue (such as a journal)—rather than a scholarly work properly speaking—have type `paratext`. These include things like front-covers, back-covers, tables of contents, and the journal itself (e.g., `https://openalex.org/W4232230324`).\n",
        "\n",
        "We also have types for `letter` , `editorial` , and `erratum` (corrections). Coverage is low on these but will improve.\n",
        "\n",
        "Other work types follow the Crossref \"type\" controlled vocabulary—see [`type_crossref`](./README.md#type_crossref).\n",
        "\n",
        "```json\n",
        "type: \"article\"\n",
        "```\n",
        "\n",
        "### `type_crossref`\n",
        "\n",
        "_String:_ Legacy type information, using Crossref's \"type\" controlled vocabulary.\n",
        "\n",
        "These are the work types that we used to use, before switching to our current system (see [`type`](./README.md#type)).\n",
        "\n",
        "You can see all possible values of Crossref's \"type\" controlled vocabulary via the Crossref api here: [`https://api.crossref.org/types`](https://api.crossref.org/types).\n",
        "\n",
        "Where possible, we just pass along Crossref's `type` value for each work. When that's impossible (eg the work isn't in Crossref), we do our best to figure out the `type` ourselves.\n",
        "\n",
        "```json\n",
        "type_crossref: \"journal-article\"\n",
        "```\n",
        "\n",
        "### `updated_date`\n",
        "\n",
        "_String:_ The last time anything in this `Work` object changed, expressed as an [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) date string (in UTC). This date is updated for _any change at all_, including increases in various counts.\n",
        "\n",
        "```json\n",
        "updated_date: \"2022-01-02T00:22:35.180390\"\n",
        "```\n",
        "\n",
        "## The `Ngram` object\n",
        "\n",
        "{% hint style=\"info\" %}\n",
        "Ngram objects are only available in the API and are not included in the [OpenAlex snapshot](./../../../download-all-data/openalex-snapshot.md).\n",
        "{% endhint %}\n",
        "\n",
        "### ngram\n",
        "\n",
        "_String:_ Group of words (or numbers, letters, etc) that exist together in the work. This can be a five-gram, four-gram, trigram, bigram, or unigram.\n",
        "\n",
        "```json\n",
        "ngram: \"energy formula into a functional\"\n",
        "```\n",
        "\n",
        "### ngram_count\n",
        "\n",
        "_Integer:_ How many times this ngram occurred in the work.\n",
        "\n",
        "```json\n",
        "ngram_count: 1\n",
        "```\n",
        "\n",
        "### ngram_tokens\n",
        "\n",
        "_Integer:_ How many tokens are in the ngram.\n",
        "\n",
        "```json\n",
        "ngram_tokens: 5\n",
        "```\n",
        "\n",
        "### term_frequency\n",
        "\n",
        "_Float:_ How often the ngram occurred in the work.\n",
        "\n",
        "**Caution:** This data was taken directly from the General Index and we've not tested `term_frequency` against actual articles. You can read about their data extraction process on the [Internet Archive](https://archive.org/details/GeneralIndex) website. If you compare `term_frequency` against articles we would [like to hear](./mailto:support@openalex.org.md) how it went!\n",
        "\n",
        "```json\n",
        "term_frequency: 0.0005452562704471102\n",
        "```\n",
        "\n",
        "## The `OpenAccess` object\n",
        "\n",
        "The `OpenAccess` object describes access options for a given work. It's only found as part of the `Work` object.\n",
        "\n",
        "### `any_repository_has_fulltext`\n",
        "\n",
        "_Boolean:_ `True` if any of this work's [`locations`](./README.md#locations) has `location.is_oa=true` and `location.source.type=repository`.\n",
        "\n",
        "Use case: researchers want to track Green OA, using a definition of \"any repository hosts this.\" OpenAlex's definition (as used in [`oa_status`](./README.md#oa_status)) doesn't support this, because as soon as there's a publisher-hosted copy (bronze, hybrid, or gold), oa_status is set to that publisher-hosted status.\n",
        "\n",
        "So there's a lot of repository-hosted content that the `oa_status` can't tell you about. Our [State of OA paper](https://peerj.com/articles/4375/) calls this \"shadowed Green.\" This feature makes it possible to track shadowed Green.\n",
        "\n",
        "```json\n",
        "any_repository_has_fulltext: true\n",
        "```\n",
        "\n",
        "### `is_oa`\n",
        "\n",
        "_Boolean:_ `True` if this work is Open Access (OA).\n",
        "\n",
        "There are [many ways to define OA](https://peerj.com/articles/4375/#literature-review). OpenAlex uses a broad definition: having a URL where you can read the fulltext of this work without needing to pay money or log in. You can use the [`locations`](./README.md#locations) and [`oa_status`](./README.md#oa_status) fields to narrow your results further, accommodating any definition of OA you like.\n",
        "\n",
        "```json\n",
        "is_oa: true\n",
        "```\n",
        "\n",
        "### `oa_status`\n",
        "\n",
        "_String:_ The Open Access (OA) status of this work. Possible values are:\n",
        "\n",
        "* **`gold`**: Published in an OA journal that is indexed by the [DOAJ](https://doaj.org/).\n",
        "* **`green`**: Toll-access on the publisher landing page, but there is a free copy in an [OA repository](https://en.wikipedia.org/wiki/Open-access_repository).\n",
        "* **`hybrid`**: Free under an [open license](https://support.unpaywall.org/support/solutions/articles/44002063718-what-is-an-oa-license-) in a toll-access journal.\n",
        "* **`bronze`**: Free to read on the publisher landing page, but without any identifiable license.\n",
        "* **`closed`**: All other articles.\n",
        "\n",
        "```json\n",
        "oa_status: \"gold\"\n",
        "```\n",
        "\n",
        "[Diamond open access](https://en.wikipedia.org/wiki/Diamond_open_access) is another more recent designation that refers to works which are free for both readers and authors. In OpenAlex, there is no \"diamond\" `oa_status`, but diamond OA works can be identified using a combination of the [`is_oa`](./README.md#is_oa) and [`apc_list.value`](./README.md#apc_list):\n",
        "\n",
        "* Get Diamond open access works: [`https://api.openalex.org/works?filter=apc_list.value:0,is_oa:true`](https://api.openalex.org/works?filter=apc_list.value:0,is_oa:true)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "# @title { run: \"auto\", vertical-output: false }\n",
        "# https://api.openalex.org/works?filter=apc_list.value:0,is_oa:true\n",
        "filter=\"apc_list.value:0,is_oa:true\" # @param {type: \"string\"}\n",
        "\n",
        "response = works_api.get_works(\n",
        "\tfilter=filter\n",
        ")\n",
        "\n",
        "df = pd.DataFrame(response.results)\n",
        "display(df)"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "numeric_df = df[['id', 'display_name'] +\n",
        "\t[col for col in df.columns if df[col].dtype in ['int64', 'float64'] and col != 'relevance_score']]\n",
        "display(numeric_df)"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "try:\n",
        "\tprompt = \"Visualize this data\" # @param {type:\"string\"}\n",
        "\tSmartDataframe(\n",
        "\t\tnumeric_df,\n",
        "\t\tconfig={\"llm\": (OpenAI(api_token=openapi_token))},\n",
        "\t).chat(prompt)\n",
        "except:\n",
        "\tif not openapi_token:\n",
        "\t\tprint(\"Error: openapi_token not set\")\n",
        "\telse:\n",
        "\t\tprint(\"Error when creating SmartDataframe\")"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### `oa_url`\n",
        "\n",
        "_String:_ The best Open Access (OA) URL for this work.\n",
        "\n",
        "Although there are [many ways to define OA](https://peerj.com/articles/4375/#literature-review), in this context an OA URL is one where you can read the fulltext of this work without needing to pay money or log in. The \"best\" such URL is the one closest to the version of record.\n",
        "\n",
        "This URL might be a direct link to a PDF, or it might be to a landing page that links to the free PDF\n",
        "\n",
        "```json\n",
        "oa_url: \"https://peerj.com/articles/4375.pdf\"\n",
        "```"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.5"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 4
}