Sources

Owners and editors of content providers in HEP Training can register and manage sources to be automatically ingested. A source consists of a URL for HEP Training to fetch, and an ingestion method which hints to HEP Training how the contents of the URL should be processed.

Once registered, a source will need to be approved by an administrator before it is active, but can be tested to see exactly what metadata HEP Training can extract from the source.

Procedure to automatically register sources¶

1. Click the ‘Sources’ tab > ‘Add source’ button on your content provider page, or if you have not yet registered a content provider, see our page Register a Content provider.

2. For URL and Ingestion Method, follow the procedure of the specific ingestion method below.

3. Select the default language

4. Tick Enabled to enable the ingestion, you can come back anytime here to disable the automatic ingestion of the source.

5. Filters: you can allow and block any value found in a property. For example, if you are ingesting an Indico category and you only want events with the word ‘workshop’ in the title, then in the ‘Allow list’, ‘Add filter condition’, select the ‘Property’: ‘Title contains’, for ‘Value’ write ‘workshop’. (To see the list of filters, see Filters’ list)

6. Register the source

7. In your source page, go to the ‘Testing’ tab, click on ‘Test Source’ wait a moment, after maximum 30 seconds it should show the expected results – see if the filters worked, or if it fetched all relevant metadata for example. If there is something unexpected, send us an email at contact.heptraining@cern.ch.

8. Once you are happy with it, you can ‘Request Approval’ in the ‘Source’ tab. An administrator will look at your source and accept or reject your request.

9. If your source gets approved, the individual resources will be ingested the day after and will need to pass under a curation process before being finally visible to the public.

Ingestion Method: Bioschemas¶

Ingests materials or events.

1. You must add the descriptve metadata (JSON-LD) in the source code of your webpage as shown in this example. Then modify the relevant fields and delete the ones which do not / cannot describe your resource. If you can’t modify your webpage source code, ask your webmaster. There is the Google Spreadsheet ingestor in case this does not work.

2. For URL, you can either add the link to your training resource OR the sitemap pointing to all the webpages being described with a JSON-LD. For example, HSF training materials are fetched because they do have a sitemap: https://hsf-training.org/training-center/sitemap.txt.

Ingestion Method: OAI-PMH¶

See whole procedure in the exchange content page.

Ingestion Method: Indico / .ics file¶

Ingests events.

Be sure that the URL is public! For URL, you can either add:

• a single ‘event’ (e.g., https://indico.cern.ch/event/1662800/),

• or the lowest ‘category’ (e.g., https://indico.cern.ch/category/2986/) to fetch all the subsequent events.

Ingestion Method: CSV file and Google Spreadsheet¶

Ingests materials.

We offer an alternative option for a user to provide a collection of training materials in a spreadsheet format, following our template.

To prepare and register a spreadsheet of materials to ingest:

1. Create a new Google Sheets spreadsheet.

2. Open the template Google Sheets spreadsheet: https://docs.google.com/spreadsheets/d/1yx2AZTPaPEU_Au2Hyb_C3TuNvH4btR9pZwCmfmTQRdc/.

3. Select everything from the Example sheet CTRL/CMD+A, copy and paste it in your empty new sheet.

4. Enter the details of your training materials under the headings provided.

5. One row corresponds to one training material, you can add as many rows/materials as you want.

6. Once your Google Sheets spreadsheet is ready, add it as a URL.

Ingestion Method: CDS videos record or search¶

Ingests materials.

For URL, you can either add:

• a ‘record’ (e.g., https://videos.cern.ch/record/3003573),

• or a search page (e.g., https://videos.cern.ch/search?page=1&size=21&q=&category=LECTURES&collections=Lectures::CERN Accelerator School) to fetch all the found records.

Ingestion Method: GitHub Repository or Page¶

Ingests materials.

For URL, you can either add:

• a GitHub page (e.g., https://hsf-training.github.io/hsf-training-unix-shell),

• or a GitHub repository (e.g., https://github.com/hsf-training/cpluspluscourse)

Other¶

For other ingestion methods, please send us an email at contact.heptraining@cern.ch to discuss how to register your content automatically in HEP Training.