Deep Earth Academy Review: Presentation on COL Data Management Efforts

Douglas Fils

dfils@oceandrilling.org (email) @fils (twitter)

Press key to advance.

View presentation at: http://goo.gl/fvsg0

Background

Open Data Approach

  • Open Data
  • Operate on MODELS (search, change, use)
  • Expose models to third party interfaces and tools
    • Google: Docs, Maps, Earth, Earth Engine, Fusion, more
    • Microsoft: PivotViewer, PowerPivot, Data Market, Excel
    • Other: Wolfram Alpha, Qwiki, etc.
Too much talk already  lets try something..  then see what we did...
						
Low Level

what does "operate on model" mean?

                Show curlCallDepth.sh script .....
            
Screen Cast
Background

What just took place?

  • USIO data is exposed as a model, not a set of services or API's
  • Like a database, this means we can "operate" on the data by means of requests.
  • It also means we can "cast" the data into different models rather easly... in the previous example we moved from RDF to KML (the Google Earth Model language)
  • RDF?
Background

RDF and Linked data

  • USIO data is exposed using the Linked Data pattern (http://www.w3.org/DesignIssues/LinkedData.html)
  • RDF is a graph model allowing relations (not just tree like relations by graph like relations)
  • We can interact with the graph via a query language (SPARQL in this case)
  • Also, URL's (URI's) now have meaning (ID) and representations (with persistence and provenance)
Background

It's a bit low level....

but show me what else it can do...

Background

A web of data?

  • Linked Data pattern encourages and enables re-use of resources.
  • Since the resources expose the model (not just the representation) it's easy for others to alter the data to fit their look and feel.
  • Current resources include (ref: http://data.oceandrilling.org/linkeddata.html)
  • Planned resources
    • Timescales
    • Core Atlas (Kathie Marsaglia)
    • much more...
non-devs 1/2

Anything for non-dev types?

non-devs 2/2

Anything for non-dev types?

Conclusion

Conclusion and Appendix

Resources

Resource List

  • Freebase
  • Refine
  • Tableau (on-line published)
  • Azure Datamarket
  • Qwiki
  • Corewall
  • Pyton Pattern
  • R
  • Linked Data Resources
  • Deep Zoom
  • CytoGraph(Web)
  • Google Resources
    • Google Fusion
    • Google Big Query
    • Google DSPL
    • Google Earth
    • Google EarthEngine
    • Google Refine
  • OData
    • PowerPivot
    • PivotViewer
    • Excel
  • WolframAlpha
APP

SPARQL interface

SPARQL end point SPARQL is a topic unto itself! (Visual SPARQL builder: http://data.oceandrilling.org/isparql/)

SELECT ?uri 
WHERE {
   ?uri skos:related <http://data.oceandrilling.org/codices/lsh/171/1049/C> .
}
		
PREFIX chronos: <http://www.chronos.org/loc-schema#>
SELECT ?uri ?long ?lat
WHERE {
     ?uri skos:related <http://data.oceandrilling.org/codices/lsh/171> .
     ?uri geo:long ?long .
     ?uri geo:lat ?lat .
}