Skip to content
Snippets Groups Projects
Commit 858140fd authored by Fiona D'Mello's avatar Fiona D'Mello
Browse files

Merge branch 'feature/move-sparql' into 'feature/update-landing-page'

Transfer SPARQL content

See merge request !10
parents 3eaf6231 bdbe8b27
No related branches found
No related tags found
3 merge requests!14Value point update,!10Transfer SPARQL content,!9Docusaurus based documentation
Pipeline #438404 passed
Showing
with 641 additions and 30 deletions
......@@ -9,6 +9,7 @@
.cache-loader
# Misc
.idea
.DS_Store
.env.local
.env.development.local
......
......@@ -6,26 +6,23 @@ stages:
docs-build:
stage: build
image: python:latest
image: node:20
script:
- pip install -U jupyter-book
- jupyter-book clean docs
- jupyter-book build docs
- ls
- npm install
- npm run build
artifacts:
paths:
- docs/_build/
- build
pages:
stage: deploy
image: busybox:latest
needs: [docs-build]
script:
- ls
- mv docs/_build/html public
- mv build public
artifacts:
paths:
- public
rules:
- if: $CI_COMMIT_BRANCH == "main"
- if: '$CI_COMMIT_TAG =~ /^\d+\.\d+(\.\d+)?(-pl\d+){0,1}$/'
---
displayed_sidebar: guidesSidebar
sidebar_position: 2
---
# SPARQL Query Language
SPARQL queries [RDF](https://www.w3.org/RDF/) graphs, which is comprises a set of semantic triple, or RDF triple or simply triple. A triple is a set of three entities that constitutes a statement about semantic data in the form of subject–predicate–object statements (e.g., "DatasetX has license CC BY 4.0").
The importance lies in the triples, not the serialization. Serialization is just a way to write the triples down.
SPARQL allows for a query to consist of triple patterns, conjunctions, disjunctions, and optional patterns as well as to perform changes on data via SPARQL updates. It also supports aggregation, subqueries, negation, creating values by expressions, extensible value testing, and constraining queries by source RDF graph. The results of SPARQL queries can be result sets or RDF graphs.
### Syntax
Before writing a SPARQL query it is important to define the namespace prefix bindings which are used int he query. A binging is a pair of a prefix name and the corresponting IRI. For instance, the following is a binding for `schema` and `rdfs` namespaces:
```
PREFIX schema: <http://schema.org/>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
```
Most forms of SPARQL query contain a set of triple patterns called a basic graph pattern. Triple patterns are like RDF triples except that each of the subject, predicate and object may be a variable. A basic graph pattern matches a subgraph of the RDF data when RDF terms from that subgraph may be substituted for the variables and the result is RDF graph equivalent to the subgraph.
### Examples
The example below demonstrates a simple query that leverages the publicatoins along with their authors. Specifically, the following query returns names and affiliation of every person participated in the creation of that resource:
```
PREFIX schema: <http://schema.org/>
SELECT *
WHERE {
?resource schema:creator ?person.
?resource schema:name ?rname.
?person schema:affiliation ?affiliation.
?person schema:name ?PersonName
}
```
The results will look like:
![](../../../static/img/SPARQL-query1-sample-results.png)
---
displayed_sidebar: guidesSidebar
sidebar_position: 1
---
### SPARQL Guide
---
displayed_sidebar: guidesSidebar
sidebar_position: 1
---
# Introduction
UnHIDE exposes the full public part of the graph through a read only SPARQL endpoint:
currently available under https://sparql.unhide.helmholtz-metadaten.de/sparql/ but in the future mainly under https://sparql.unhide.helmholtz-metadaten.de/.
<p style={{ textAlign: "center" }}>
<img
align="right"
width="250"
src="https://community.openlinksw.com/uploads/default/original/1X/e92dc4ebb5a2f272de53b2dff1f928f5d940c1f3.png"
/>
</p>
UNHIDE runs a [Virtuoso SPARQL](https://vos.openlinksw.com/owiki/wiki/VOS/VOSSPARQL) Endpoint which could be used to query the unHIDE knowledge graph using the [SPARQL Query Language](https://www.w3.org/TR/sparql11-query/) — a W3C open standard which can be used to express queries across diverse data sources, whether the data is stored natively as RDF.
The server is running as part of the larger deployment of unHIDE, as a part of the Docker compose file.
<!-- ![](https://community.openlinksw.com/uploads/default/original/1X/e92dc4ebb5a2f272de53b2dff1f928f5d940c1f3.png =400x)
-->
---
displayed_sidebar: guidesSidebar
sidebar_position: 3
---
# Virtuoso SPARQL Endpoint
[Virtuoso universal server](https://docs.openlinksw.com/virtuoso/) provides the [SPARQL 1.1 protocols](http://www.w3.org/TR/sparql11-protocol/) for query and update as well as the SPARQL Graph Store protocol. Virtuoso reserves the path `/sparql/` and a synonym path `/SPARQL/` for SPARQL service. Virtuoso’s built-in SPARQL Query Service functionality delivers a variety of capabilities:
- Supports the HTTP-based SPARQL Query Service Protocol
- Supports Federated SPARQL (i.e. supports `SERVICE` clause that identifies an external SPARQL endpoint upon which to execute a sub-query — as one option for executing federated queries over SPARQL endpoints from the [LOD Cloud](https://lod-cloud.net/).
- Supports optional de-referencing of variables and constants in the body of a query, thereby incorporating content crawling over external data sources into the query solution production pipeline.
- Support for a variety of query types including CONSTRUCT, ASK, DESCRIBE.
- Supports a wide variety of query solution document types including (among others) HTML, CSV, JSON, and XML for SELECT Queries, and RDF-Turtle, RDF-N-Triples, JSON-LD, and RDF-XML for DESCRIBE and CONSTRUCT Queries.
- Includes a query optimaization service that produces the SPARQL query after parsing, optimization and converting back into SPARQL.
- Adds Extract, Transform, and Load operations to the query solution production pipeline through built-in integration with Virtuoso RDF Transformation Middleware (or [RDFizer](https://github.com/SDM-TIB/SDM-RDFizer)), commonly referred to as the Sponger — which also includes cache invalidation functionality.
- Supports HTTP Content Negotiation ("conneg"), both Basic and Dynamic via QoS algorithms.
- Also exploitable by ODBC, JDBC, ADO.NET, OLE DB, and XMLA applications as an open standards-based extension to SQL.
### Virtuoso SPARQL Query Service
Both `GET` and `POST` requests are supported by both server and client. The server recognizes the `Accept:` line of the request header to find MIME types preferred by the connected client and adjust the output mode of the response. The client chooses between GET and POST automatically, based on the length of the query text. More information can be found [here](https://vos.openlinksw.com/owiki/wiki/VOS/VOSSparqlProtocol).
### Virtuoso Facet Browser
The Virtuoso Facet Browser UI allows users to save Faceted View and iSPARQL queries as "Featured" queries for future reference.
![](../../../static/img/fct.png)
Use the Faceted Browser Search to search for information on "HZDR":
![](../../../static/img/fct-hzdr.png)
Results of the following form should be returned for the resource being fetched:
![](../../../static/img/fct-hzdr-results.png)
where users can find all entities that are related to `HZDR` and their title, type, etc. Below is an example of one of these entities:
![](../../../static/img/fct-hzdr-results-ex.png)
Additionally, users can export these results in various formats, including csv, HTML, RDF (N-triples), RDF (turtule), etc.
Here is the results exported as RDF/XML:
```xml
<?xml version="1.0" encoding="utf-8" ?>
<rdf:RDF
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
xmlns:schema="http://schema.org/" >
<rdf:Description rdf:about="https://orcid.org/0000-0001-7990-9564">
<rdf:type rdf:resource="http://schema.org/Person" />
<schema:affiliation>HZDR</schema:affiliation>
<schema:name>Pausch, Richard</schema:name>
</rdf:Description>
</rdf:RDF>
```
**Entity Label Lookup** tab: In the Label auto-complete text box of the Entity Label Lookup tab, enter the name of an rdfs label to be Described, then select a URI from the list of available Labels to obtain a description of the URI.
![](../../../static/img/fct-lookup.png)
**Entity URI Lookup** tab: In the URI auto-complete text box of the Entity URI Lookup tab enter the name URI to be Described, then select a URI from the list of available Labels to obtain a description of the URI.
![](../../../static/img/fct-uri-lookup.png)
---
displayed_sidebar: guidesSidebar
sidebar_position: 3
---
# /categories
This API provides the user information on the basic categories by the which the information in the HKG is grouped, and the total number of records found for each of this categories, respectively.
### Method
GET
### Search Params
None
### URL
[https://search.unhide.helmholtz-metadaten.de/api/categories](https://search.unhide.helmholtz-metadaten.de/api/categories)
### Response Model
```
[
{
id: string,
text: string,
count: number
}
]
```
where, `id` is the identifier of the category, `text` is the name of the category to be displayed on the UI, and `count` presents the total number of records available for the given category
#### Example
```
[
{
id: "datasets",
text: "Datasets",
count: 411935,
},
{
id: "software",
text: "Software",
count: 3521,
},
{
id: "documents",
text: "Documents",
count: 1679342,
},
{
id: "instruments",
text: "Instruments",
count: 0,
},
{
id: "experts",
text: "Experts",
count: 229601,
},
{
id: "institutions",
text: "Institutions",
count: 64472,
},
{
id: "trainings",
text: "Trainings",
count: 667,
},
{
id: "projects",
text: "Projects",
count: 0,
},
]
```
---
displayed_sidebar: guidesSidebar
sidebar_position: 2
---
# /hello
This API is currently used to test that the server is up and available. It will eventually be pointing to this documentation.
### Method
GET
### Search Params
None
### URL
[https://search.unhide.helmholtz-metadaten.de/api/hello](https://search.unhide.helmholtz-metadaten.de/api/hello)
### Response Model
```
{
"message": string
}
```
#### Example
```
{
"message": "Hello from Next.js!"
}
```
---
displayed_sidebar: guidesSidebar
sidebar_position: 3
---
### Web API GUIDE
---
displayed_sidebar: guidesSidebar
sidebar_position: 1
---
# Introduction
The web APIs related to UnHIDE are made available at [https://search.unhide.helmholtz-metadaten.de/api/[endpoint]](https://search.unhide.helmholtz-metadaten.de/api/hello). There are currently four APIs defined, the details of which are covered in the following sections
---
displayed_sidebar: guidesSidebar
sidebar_position: 5
---
# /source
This API provides the user the JSON-LD object containing information of a record for the given ID.
### Method
GET
### Search Params
`id` - the identifier value of the record of interest
#### Examples
`id=e6be83435b277f34cab97adbefbe7b03` Or
`id=https%3A%2F%2Fror.org%2F04t3en479`
### URL
[https://search.unhide.helmholtz-metadaten.de/api/source?id=](https://search.unhide.helmholtz-metadaten.de/api/source?id=)
### Response Model
```
{
@type: string,
...
}
```
where, `@type` is the type of the metadata record identified,
additionally, all information that is avaliable in the record, depending on its type is sent across as part of the response object
#### Examples
For `id=https%3A%2F%2Forcid.org%2F0000-0001-9827-4429`
```
{
"@id": "https://orcid.org/0000-0001-9827-4429",
"@type": "Person",
"affiliation": {
"@id": "https://ror.org/03v76x132",
"@type": "Organization",
"name": "Yale University"
},
"familyName": "Liang",
"givenName": "Yu",
"name": "Yu Liang"
}
```
For `id=https%3A%2F%2Fror.org%2F04t3en479`
```
{
"@id": "https://ror.org/04t3en479",
"@type": "Organization",
"name": "Karlsruhe Institute of Technology"
}
```
---
displayed_sidebar: guidesSidebar
sidebar_position: 3
---
# /search
This API provides the user the search results for a given search query.
### Method
GET
### Search Params
`searchText` - term that is being searched for (string)
`start` - row of record to start fetching from (optional; number; defaults to 0)
`rows` - number of rows to be fetched in a call (optional; number; defaults to 10)
`documentType` - the identifier of the category of interest (optional; string; see [/categories](/docs/guides/web-api/categories-api#example) for possible values)
`includeFacet` - flag to indicate whether dynamic filters are required for a given search (optional; boolean)
`facetType` - identifier of the dynamic filter (optional, works with `facetName`; string; see [data model](/docs//about/data-model/index.md#filters) for possible values)
`facetName` - value of the dynamic filter chosen (optional, works with `facetType`; string; this depends on the possible values made available for this filter type)
### URL
[https://search.unhide.helmholtz-metadaten.de/api/search?[searchParams]](https://search.unhide.helmholtz-metadaten.de/api/search?)
### Response Model
```
{
"records": [
{
"id": string,
"type": string,
"keys": [ string ],
...
"recordConfig": {
"index_id": string,
"_version_": string,
"indexed_ts": timestamp string
}
},
],
"totalCount": number,
"categoryCounts": {
{categoryid}: count,
...
},
"filters": [
{
"filterName": string,
"itemName": string,
"itemCount": number
},
...
]
}
```
where,
`records` is a list of records fetched based on the search params. Each record object can have different key-value pairs depending upon the category of the data under interest.
`totalCount` is the total number of records available for the given search
`categoryCounts` presents the total number of records available for the given category and search,
`filters` provides a list of filter objects if the `facetType` and `facetName` are supplied when `includeFacet` is set to true for a given search call.
#### Example
For the following search params:
`searchText` - "Forschungszentrum Jülich"
`documentType` - "documents"
`facetType` - "txt_contributor"
`facetName` - "Angst, Manuel"
`facetType` - "txt_contributor"
`facetName` - "Zorn, Reiner"
`facetType` - "txt_publisher"
`facetName` - "Forschungszentrum Jülich GmbH, JCNS / RWTH Aachen / Univ. Münster"
`facetType` - "txt_keywords"
`facetName` - "info:eu-repo/classification/ddc/600"
```
{
"records": [
{
"id": "https://juser.fz-juelich.de/record/22096",
"type": "DigitalDocument",
"txt_contributor": ["Angst, Manuel", "Brückel, Thomas", "Roth, G", "Zorn, Reiner"],
"txt_creator": ["Brückel, Thomas"],
"txt_identifier": [
"https://juser.fz-juelich.de/record/22096",
"https://juser.fz-juelich.de/search?p=id:%22PreJuSER-22096%22"
],
"txt_keywords": [
"info:eu-repo/classification/ddc/530",
"info:eu-repo/classification/ddc/600"
],
"txt_license": ["info:eu-repo/semantics/openAccess"],
"name": "01: Introduction: Neutron Scattering in Contemporary Research",
"txt_provider": [
"JCNS Neutron Lab Course / Th. Brückel, D. Richter, G. Roth, R. Zorn (Eds.) ; Schriften des Forschungszentrums Jülich / Reihe Schlüsseltechnologien / Key Technologies, Vol. 39, Forschungszentrum Jülich GmbH, 2012; 978-3-89336-789-4<br/>JCNS Neutron Laborator Course, Jülich/München, Germany, 2012-09-03 - 2012-09-14"
],
"txt_publisher": ["Forschungszentrum Jülich GmbH, JCNS / RWTH Aachen / Univ. Münster"],
"txt_sameAs": ["[info:eu-repo, semantics, altIdentifier, hdl, 2128, 4740]"],
"keys": [
"id",
"type",
"txt_contributor",
"txt_creator",
"txt_identifier",
"txt_keywords",
"txt_license",
"name",
"txt_provider",
"txt_publisher",
"txt_sameAs"
],
"recordConfig": {
"index_id": "3915592a-f9cd-4932-a048-7aa1019d3a67",
"_version_": "1778867342397145088",
"indexed_ts": "2023-10-04T22:54:56.738Z"
}
},
{
"id": "https://juser.fz-juelich.de/record/19548",
"type": "DigitalDocument",
"txt_contributor": ["Angst, Manuel", "Brückel, Thomas", "Roth, G", "Zorn, Reiner"],
"txt_creator": ["Frielinghaus, Henrich"],
"txt_identifier": [
"https://juser.fz-juelich.de/record/19548",
"https://juser.fz-juelich.de/search?p=id:%22PreJuSER-19548%22"
],
"txt_keywords": [
"info:eu-repo/classification/ddc/530",
"info:eu-repo/classification/ddc/600"
],
"txt_license": ["info:eu-repo/semantics/openAccess"],
"name": "05: Nanostructures Investigated by Small Angle Neutron Scattering",
"txt_provider": [
"JCNS Neutron Lab Course / Th. Brückel, D. Richter, G. Roth, R. Zorn (Eds.) ; Schriften des Forschungszentrums Jülich / Reihe Schlüsseltechnologien / Key Technologies, Vol. 39, Forschungszentrum Jülich GmbH, 2012; 978-3-89336-789-4<br/>JCNS Neutron Laborator Course, Jülich/München, Germany, 2012-09-03 - 2012-09-14"
],
"txt_publisher": ["Forschungszentrum Jülich GmbH, JCNS / RWTH Aachen / Univ. Münster"],
"txt_sameAs": ["[info:eu-repo, semantics, altIdentifier, isbn, 978-3-89336-789-4]"],
"keys": [
"id",
"type",
"txt_contributor",
"txt_creator",
"txt_identifier",
"txt_keywords",
"txt_license",
"name",
"txt_provider",
"txt_publisher",
"txt_sameAs"
],
"recordConfig": {
"index_id": "4e71f085-59f4-4dc0-8ac5-4f07a2c8af15",
"_version_": "1778856255287197696",
"indexed_ts": "2023-10-04T19:58:43.246Z"
}
}
],
"totalCount": 2,
"categoryCounts": {
"experts": 0,
"documents": 2,
"datasets": 0,
"institutions": 0,
"trainings": 0,
"projects": 0,
"software": 0,
"instruments": 0
},
"filters": [
{
"filterName": "txt_keywords",
"itemName": "info:eu-repo/classification/ddc/530",
"itemCount": 2
},
{
"filterName": "txt_keywords",
"itemName": "info:eu-repo/classification/ddc/600",
"itemCount": 2
},
{
"filterName": "txt_contributor",
"itemName": "Angst, Manuel",
"itemCount": 2
},
{
"filterName": "txt_contributor",
"itemName": "Brückel, Thomas",
"itemCount": 2
},
{ "filterName": "txt_contributor", "itemName": "Roth, G", "itemCount": 2 },
{ "filterName": "txt_contributor", "itemName": "Zorn, Reiner", "itemCount": 2 },
{
"filterName": "txt_license",
"itemName": "info:eu-repo/semantics/openAccess",
"itemCount": 2
},
{
"filterName": "txt_provider",
"itemName": "JCNS Neutron Lab Course / Th. Brückel, D. Richter, G. Roth, R. Zorn (Eds.) ; Schriften des Forschungszentrums Jülich / Reihe Schlüsseltechnologien / Key Technologies, Vol. 39, Forschungszentrum Jülich GmbH, 2012; 978-3-89336-789-4<br/>JCNS Neutron Laborator Course, Jülich/München, Germany, 2012-09-03 - 2012-09-14",
"itemCount": 2
},
{
"filterName": "txt_publisher",
"itemName": "Forschungszentrum Jülich GmbH, JCNS / RWTH Aachen / Univ. Münster",
"itemCount": 2
}
]
}
```
---
displayed_sidebar: guidesSidebar
sidebar_position: 2
---
## WEB UI guide
---
displayed_sidebar: guidesSidebar
sidebar_position: 1
---
# Introduction
The web UI is primarily intended to search the HKG. Thus the most primary features that we provide currently include a search bar, a representation of basic categories for easy access to relevant information and the ability to filter information pertaining a specific category with appropriate filters.
Additionally, we allow users a look at every meta-data records JSON-LD source. We also make it possible for the user to navigate to relevant sources when the source information in the form of hyperlink-able IDs, exist for any given piece of information. There are few more features that will be briefly mentioned in the following sections.
---
displayed_sidebar: guidesSidebar
sidebar_position: 2
---
# Primary Features
## Categorized Search
As mentioned in the description of our [data model](/docs/about/data-model/index.md), we combine different types of harvested metadata based on their [schema.org](https://schema.org) type, into 8 categories. This feature allows users to easily search for their records of interest based on how we model our data. Depending upon what category is of interest to the user, the user can directly access records filtered based on their categories via the clickable buttons, that also show the total count of records for the said category, from the landing page.
![Image of unhide landing page categories for search](../../../static/img/unhide-landing-categories.png "Unhide landing page categories for search")
The same categorization is available in the results page as a tab as shown in the image below
![Image of unhide results page categories tab](../../../static/img/unhide-results-categories.png "Unhide unhide results page categories tab")
## Search Bar
The user can also search for records of their interest via the search bar. The search bar also features a random set of queries that the user can try if they like. In the following example, a user is seen searching the HKG for records related to the term "Molecular Dynamics".
![Image of unhide landing page search](../../../static/img/unhide-landing-search.png "Unhide landing page search")
The results are shown in the results view as a list of records where earch record is displayed within a card. A maximum of 10 records for a search are shown at a time, but the results are paginated so connected results are readily available to the user. The user can also use the category tab above the results to browse through records of that category for the users search query
![Image of unhide search results](../../../static/img/unhide-results-search.png "Unhide search results")
The search bar is always readily available to user regardless of the view they are in.
## Dynamic Filters
Depending on the category of results shown, the data could also be filtered if there are filters available for the category and the given search term. The different available filters are shown to the left of the list of resulting records.
![Image of unhide search filters](../../../static/img/unhide-results-filters.png "Unhide search filters")
Each set of filters can be further filtered using a searchbar specific to the filter.
![Image of unhide filtered search filters](../../../static/img/unhide-filters-filtered.png "Unhide filtered search filters")
Filters from a single or multiple set can be simultaneously selected. Please note that, each time a specific filter is chosen, the user would be refining their search and as such the filters could be dynamically updated.
The selected filters are shown as tags above the results and can be individually or collectively unselected.
![Image of unhide search filter tags](../../../static/img/unhide-filter-tags.png "Unhide search filter tags")
---
displayed_sidebar: guidesSidebar
sidebar_position: 3
---
# Other features
## JSON-LD source
For each metadata record shown in the results list, at the bottom of each card, the user can find a link to all information in the record displayed in its JSON-LD object form. This feature allows users to look for details of the record on a more granular level and might help in case a piece of information they are looking for exists as part of the record but is not explicitly shown in the record card of the web view.
## Hyperlinked metadata
In cases where a particular piece of information in a metadata record has a web address and can be navigated to, this piece of information is made clickable - which takes the user to the web page connected to the information.
## Responsive UI
The web UI application is available for usage on mobile, iPad, laptop and desktop views. This makes the application accessible for users in most widely used devices.
File moved
......@@ -23,7 +23,11 @@ const sidebars: SidebarsConfig = {
type: "generated-index",
},
collapsed: true,
items: ["guides/web-ui/intro"],
items: [
"guides/web-ui/introduction",
"guides/web-ui/primary-features",
"guides/web-ui/secondary-features",
],
},
{
type: "category",
......@@ -32,7 +36,13 @@ const sidebars: SidebarsConfig = {
type: "generated-index",
},
collapsed: true,
items: ["guides/web-api/intro"],
items: [
"guides/web-api/introduction",
"guides/web-api/hello-api",
"guides/web-api/categories-api",
"guides/web-api/search-api",
"guides/web-api/json-ld-source-api",
],
},
{
type: "category",
......@@ -41,7 +51,11 @@ const sidebars: SidebarsConfig = {
type: "generated-index",
},
collapsed: true,
items: ["guides/sparql/intro"],
items: [
"guides/sparql/introduction",
"guides/sparql/about-sparql",
"guides/sparql/virtuoso-sparql",
],
},
],
communitySidebar: [{ type: "autogenerated", dirName: "community" }],
......
static/img/SPARQL-query1-sample-results.png

93.3 KiB

static/img/fct-hzdr-results-ex.png

36.4 KiB

0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment