LOD for applications
Using the Lobid API
Pascal Christoph /
@dr0ide & Fabian Steeg / @fsteeg
Linked
Open Data, Hochschulbibliothekszentrum NRW (hbz)
SWIB15, Hamburg, 2015-11-24
This presentation:
http://hbz.github.io/slides/swib-15
ALA Archives, ca. 1925
http://imagesearchnew.library.illinois.edu/cdm/singleitem/collection/alaposters/id/23/rec/2
Software
The thing on which all professions and occupations depend
Including library work
Libraries are Software
"Libraries are Software" by @codyh — Good short essay
for dev and non-dev librarians http://t.co/Y9YLmTaZj0
pic.twitter.com/iVGKFaBql8
— Tim Spalding (@librarythingtim) September
17, 2015
LOD for applications
Using the Lobid API
Application Programming Interface
Why use an API?
LOD for applications
= building software
APIs make software development manageable
modular software
stable applications
flexible data sources
Lobid API data sources and formats
API decouples applications from specific data sources, formats, and systems
Using the Lobid API
Using APIs?
What does that even mean?
For Web APIs:
opening URLs!
Just like opening a web site
but we want structured data
How-to
Click the center of screenshots to switch to live demo(You should have a JSON viewer plugin installed in your Browser for readable output)
We can edit and re-open the URL in the browser bar
We're using the API
Same URL, other usage:
Call URL from command line
Bulk downloads
curl -H "Accept: application/ld+json" -H "Accept-Encoding: gzip"For or arbitrary queries
# All holdings of owner with ISIL DE-6:
"http://lobid.org/resource?owner=DE-6&scroll=true"# Or new titles with GND subject 4055382-6 since 2015-10-23:
"http://lobid.org/resource?subject=4055382-6&scroll=20151023"# And redirect data into a file:
> "resources-4055382-20151023.gz"(this should all be in a single line, using one URL only)
Local data, offline usage, still API
Application:
answer a single question
How many
libraries are there in Germany?
10.000?
.@fsteeg "10.200
[Bibliotheken] gibt es insgesamt". Sind eher doppelt so
viel, wenn man die Pfarrbibliotheken dazu zählt.
— Adrian Pohl (@acka47) May
29, 2015
20.000?
In Deutschland gibt es 19.347 Bibliotheken http://t.co/qPcY3Kh7o7 .
Datenquelle: dbs und ZDB-Sigeldatei /via @lobidOrg API
— dr0ide (@dr0ide) June
10, 2015
How-to
(summary)
How many libraries are there in Germany?
13.279
| Look at a sample organisation | http://beta.lobid.org/organisations/DE-206H |
| Find fields to query | type AND location.address.addressCountry |
| Create query | http://beta.lobid.org/organisations/search?q=type:Library+AND+location.address.addressCountry:Germany |
| Find answer in response | hits.total: 13.279 (19.11.15) |
19.347 vs. 13.279
LOD for applications
Using the Lobid API
Using the Lobid API
Improved queries
@lobidOrg @InspektorHicks
Um genau zu sein: da sind auch Goethe Institute im Ausland mit
dabei usw. Siehe http://t.co/wbpiGZl8OC
— dr0ide (@dr0ide) June
10, 2015
Improved data
Take into account status of DBS entries
Usage → Improvement
Application:
visualize data on a map
Libraries in Hamburg
How-to
(summary)
Build an interactive map
| Look at a sample organisation | http://beta.lobid.org/organisations/DE-206H |
| Find field to query | type AND location.address.addressLocality |
| Create query | http://beta.lobid.org/organisations/search?q=type:Library+AND+location.address.addressLocality:Hamburg |
| Embed some Leaflet.js Javascript code in an HTML file which calls query above and uses response fields | location.geo.lat and location.geo.lon |
| Open HTML file in browser | demo.html, source |
Application:
NWBib
NWBib: regional bibliography for German state North Rhine-Westfalia (NRW)
Catalogued by the 3 NRW-Landesbibliotheken
Presentation by hbz, based on Lobid API
NWBib
(summary)
Complex application with map visualization and browsing
| Landesbibliotheken catalog subject locations, e.g. Köln | http://lobid.org/nwbib/HT016558942 |
| We get geo coordinates for these labels from Wikidata (so we can search by geo location) | http://lobid.org/nwbib |
| We get labels for these geo coordinates from Wikidata (so we can show a label if we search by geo location) |
http://lobid.org/nwbib/search?location=50.942222,6.957778 |
| Display them on a searchable map, basically as in previous demo, with clusters as visualization | http://lobid.org/nwbib/search?q=braunkohle |
Application:
OpenRefine
A free, open source, power tool
for working with messy data.
Spreadsheet
Cleanup
Transformation
Reconciliation
(with external APIs)
Use case: user request
https://github.com/hbz/lobid-organisations/issues/55
| dbs-id | bibliothek | plz |
|---|---|---|
| Stadtbücherei Wernau | 73249 | |
| WG548 | Stadtbibliothek Stavenhagen | 17153 |
| Stadt- und Schulbücherei Gunzenhausen | 91710 | |
| WH759 | Gemeindebibliothek Hohenleipisch | 4934 |
Wanted: uniform identifiers
How-to
(summary)
OpenRefine reconciliation
| Get OpenRefine | https://github.com/OpenRefine/OpenRefine/releases |
| Follow instructions | https://github.com/hbz/lobid-organisations/issues/55\#issuecomment-129771979 |
Lessons learned
Don't lose useful data for applications in LOD transformation
Structure data to make it useful, like for queries and processing
Integrate into existing workflows and open tools, like OpenRefine
Let applications drive API and data design
avoid premature abstraction
support actual use cases before generalizing
Do usable before reusable.
Do usable before reusable.
— Einar W. Høst (@einarwh) August
21, 2015
Having other folks use your stuff makes
your stuff better!
@edsu Guess what - it turns
out having other folks use your stuff makes your stuff better!
Software and collections too!
— Andy Jackson (@anjacks0n) July
30, 2015
Progress
Building
Using
Improving
How did we built the API?
Metafacture
(a Java toolkit for stream-based
library metadata processing)
Elasticsearch
(a Lucene-based search server,
something like SOLR, we use it with Java)
Playframework
(a Web application framework,
something like Rails or Django, we use it with Java)
Java programming
with open source tools!
But what about
Linked Data
Semantic Web
in Libraries
SWIB
?
(our structured data is JSON-LD, RDF-compatible)
a technological solution, one of many that
might fit the real goals
I'm so over linked data/semantic web
as a goal. Its a technological solution, one of many that might
fit the real goals/outcomes
— Euan Cochrane (@euanc) June
2, 2015
to make the product better for our users
"No matter what our job titles, our jobs are all the same —
to make the product better for our users." http://t.co/Rz8CnWvlbm
— F. Steeg (@fsteeg) February
25, 2015
Software
in Libraries
SWIB
!
So what?
Build APIs
provide infrastructure for software in libraries
make the great work of cataloguers available
for all kinds
of use cases
Libraries need APIs
.@fsteeg Yep. Libraries
need APIs, not web portals.
— Ralf Claussnitzer (@claussni) July
29, 2015
Empower yourself and others
to use your data
to build new applications
to improve existing applications
Not by itself
An API and some large-but-finite amount of
labor
Realizing that I say "it
wouldn't be difficult to X" when I mean "there's
an API and some large-but-finite amount of labor could X."
— Ted Underwood (@Ted_Underwood) September
2, 2015
Provide infrastructure for
software in libraries
Empower yourself and others
(Openness)
| Technically open: code is open source; organizationally open: issues, continuous integration, code reviews, wiki on GitHub | http://github.com/hbz/lobid, http://github.com/hbz/nwbib |
| API is open, use at your will | http://lobid.org, http://beta.lobid.org/organisations |
We're very happy about usage, feedback, and contributions on all of these levels.
© Jaymi Heimbuch
Original content by the authors of these slides licensed under CC-BY-4.0