Half a croissant, on a plate, with a sign in front of it saying '50c'
h a l f b a k e r y
Faster than a stationary bullet.

idea: add, search, annotate, link, view, overview, recent, by name, random

meta: news, help, about, links, report a problem

account: browse anonymously, or get an account and write.



Data Browser

A browser that makes hyperlinks between data records meaningful to make. (Internet of Data)
  [vote for,

The idea that any record in any database could have a "url" to any other record in any other database ("Internet of Data") may be useful in many situations, including the creation of semantic web.

Given a good popular data browser, many would probably care to open their databases for public access with default database ports, like 5432, or 27017, for data consumption, and so, we could start using hyperlinks like:

postgresql :// www.example.com:5432/example/topics/123

The database programming languages like (PL/SQL, PSL, etc.) would come back to relevance, allowing the flexibility to define the added business logic and functionality, which these days usually is provided by building an API on top of databases.

Today, while most of the web pages on ports 80/443 are open to public, traditionally, most records in databases are behind an authentication, and thus, people are not inclined to make such links (they are inclined to scrape the web rather than properly use databases).

Given an existence and wide knowledge of a good data browser (with ease of use and powerful analytics capabilities on local computers), it is reasonable to expect that people would care to make proper public access to them, and create a web of data.

Inyuki, Oct 22 2017

Memorandum http://www.kurzweil...ic-computer-network
Some talk about where it would all go. [mylodon, Oct 22 2017]

http://lod-cloud.net/ http://lod-cloud.net/
A great big list of open databases queryable by SPARQL If the WWW is all the linked web-pages in the world, this represents a formative top-down view of all the linked-data-web as it currently stands. [zen_tom, Oct 23 2017]

SPARQL https://en.wikipedia.org/wiki/SPARQL
A query language for linked data supporting federated queries, amongst other things. [zen_tom, Oct 23 2017]

TBL discussing linked data 10+ years ago. https://www.w3.org/...ues/LinkedData.html
[zen_tom, Oct 23 2017]

Python Library: rdflib https://github.com/RDFLib/rdflib
Full sparql support in case you want to write your own linked-data browser. [zen_tom, Oct 23 2017]

Franz: Gruff https://franz.com/agraph/gruff/
RDF/Triple-Store browser that supports federated queries - nice thing about this one is the direct visualisation view, though has a fairly hefty license tag. [zen_tom, Oct 23 2017]


       The only problem is getting people to standardize on something or to open databases to something. It's a very untrusted world out there. For instance PDF's are how most people exchange very important information, and those things are two steps from a pcx file, and you can't even palette cycle them.   

       If you want to go one better, you get CSV.   

       It's time to re-read "The Memorandum for members and affiliates of the intergalactic computer network."
mylodon, Oct 22 2017

       The difference between this and say a SQL over Telnet is .... ?
FlyingToaster, Oct 23 2017

       [FlyingToaster] This would have to implement some kind of generic query language, and ready-made queries, and maybe visualization capabilities not seen in command line. Communal databases like HB would be way more interactive :)
Inyuki, Oct 23 2017

       I was under the impression SQL is a "generic query language".   

       There is any number of tools today that will attempt to create analytically useful data from queries.   

       SQL (at least the variants I'm familiar with), already supports database.table syntax in queries, so that queries can be made across databases, and there's no particular reason why the database part of the name cannot be a URL.   

       Modern SQL also returns data in XML and can be made to return json   

       So the principal idea here is to open the databases to querying and to have a DNS like super table of databases?
theircompetitor, Oct 23 2017

       Exactly! [theircompetitor]
Inyuki, Oct 23 2017

       I thought XQuery would be more prevalent. I'll probably also be thinking that I thought GraphQL would be more prevalent, in the future.
Ian Tindale, Oct 23 2017

       JSONPath, too, also, as well.
Ian Tindale, Oct 23 2017

       I guess there isn't much call for standards in embedding of live objects. Can't think of much call for one really. A gif url with an expiry of 10 minutes would give weather and guaranteed copyright.   

       Will be interesting to see what happens when everything gets block-chained to avoid e.g. fake databases.
bigsleep, Oct 23 2017

       How would block-chaining prevent the inclusion of fakes I wonder?
RayfordSteele, Oct 23 2017

       Psst! You're describing almost exactly corresponds with the current state-of-the-art in linked-data, as originally suggested by Tim Berners Lee around 2006.   

       The analogue for html is rdf, and the world-wide-linked- data-web is implemented using servers running an rdf-native query language called SPARQL. In the public domain, there is an open, queryable version of wikipedia that's available via federated SPARQL query, plus a host of other life-science, language, and other domain-specific repositories.   

       Where I work, we use it to keep track of enterprise architecture and organisational components (applications, racks, servers, facilities, staff, management, departments, audits, regulations, risk-events, market- events, and other stuff) in a federated set of open (or shut, depending on the security context) RDF databases accessed over the network by SPARQL.   

       What's neat about this is that while each individual application owner can publish an RDF version of their data (or allow us to generate one) it's a relatively light-touch-task.   

       However, once done, it means it's finally possible to situationally query what business people will be affected due to a server outage in a particular building, identify which of their clients are most likely to be affected negatively, and send a report containing the law offices of the most litigious of these to the legal department as a heads-up.   

       Previously, that query would have been impossible, but with rdf data collected from applications, their data- flows between one another, the it hardware estate, organisational and managerial trees, business process models, client lists, business activity, KYC data and legal, and the ability to stitch all these together into a meaningful line of dependency, it becomes possible.   

       We've written a few browsers, but to date, my favourite RDF browser tool is something called Gruff, by a company called Franz. Though, for real-in-depth analysis, opensource libraries (like rdflib for example) provide the most flexibility.
zen_tom, Oct 23 2017

       //How would block-chaining prevent the inclusion of fakes I wonder?//   

       Well if take a simple case like RDF, its just pass around until it changes and then you have traceback to where it changed i.e. cryptographically secure data traceback. This really is going to be fun for people who nick data or just invent it.   

       //my favourite RDF browser tool is something called Gruff//   

       At the end of the day though, what this idea is talking about is a bulk export format so that complex queries can be performed that otherwise would only work as stored procedures.   

       Working with SQL in the early days without stored procedures was hell.
bigsleep, Oct 23 2017

       [zen_tom], you correctly noticed it, Tim Berners-Lee highlighted its importance many years ago. So, why, when so many are talking about IoT, few are aware of a bigger idea of Linked Data? Maybe it's not properly marketed, maybe we should call it not "Linked Data", but "IoD"? But,.. just maybe, the problem is that we don't have that Data Browser?
Inyuki, Oct 23 2017

       I think it's a few different things at once - and I concede the point, a key element of that is the browser - if you can democratise the experience, make it simpler for people to work with linked data, the result will be more users, more take-up of the idea.   

       Having seen things like Gruff, and other subject- predicate-object node-driven tools (lodlive etc), the dynamically constructed graph seems to be a common model. I've experimented with projecting these node- graphs into 3d using (for example) minecraft as a browser - which was fun for a bit, but the data needs to find ways to organise itself in ways that the user can control, and (for example) minecraft was too static for that to happen.   

       Another idea we have for our own visualisation tool is setting different levels of aggregation, where at a given zoom level, properties of key nodes are considered to be parts of a 'cloud' representing a single entity, which splits apart at differing levels of enzoomedness. The trick seems to be tagging different property sets as having differing levels of specificity.
zen_tom, Oct 23 2017

       //The trick seems to be tagging different property sets as having differing levels of specificity.//   

       Easily done during the import, or just have the protocol return a constant for a data item e.g. the url table name.   

       Either way, you're still going to need to import a huge amount of data and employ a lot of budget creating hand-waving.
bigsleep, Oct 23 2017

       It is not really a thing, it seems to me, that requires a “browser”. A web browser as we know it is mainly a thing that displays marked-up hypertext. An unpaged displayer of text, with in-flow graphics (but not out of flow, or anchored, in any useful sense, even still today). Basically something that displays editorial matter in the same way that I’d see it on screen in QuarkXpress or InDesign before it becomes sent to repro for printing on paper.   

       Do we have ‘browsers’ for SQL? No we don’t. All we’d see is unexciting reams of data in tables, with no attempt at typography, layout or good design sense. SQL tables also don’t count as hypertext, whereas this stuff in this posted idea would. You’d have to ‘turn it into’ something visual to ‘browse’ it, in which case it can use the conventional html5 browser as the endpoint (perhaps via a javascript library to consume the data and another to display it on canvas, if you can ever find any javascript that actually works once it is copied and pasted in).
Ian Tindale, Oct 23 2017

       There *are* data visualisation tools, things like Tableau, Qlikview, d3 and others. Common things you might want to do with this kind of data is explore shapes, determine clusters, things that are similar to other things in unusual or unexpected dimensions - which suggests network diagrams of nodes connected by edges arranged in such a way to demonstrate the structure. Force Directed Graphs are one common layout device that might be a data- browser default setting, but I think such a browser should auto-detect common network shapes such as trees, directed-flow-networks, clustered arrangements etc and be able to automatically pick a layout that best fits the content.   

       If I'm looking at a directed flow, ideally, I want all the sources on the left, and all the final sinks on the right, (or you might prefer up- down, or right-left orientations, or a clockwise/anti-clockwise view, whatever) with some intelligent placement of intermediate nodes such that the strongest flows are evident, Sankey style. This is one such layout option that could be inferred from a dataset and offered to a casual user.   

       The point being - in our very small department at work - one line we're exploring is just this idea - common, generic methods for visualising linked- data queries in the browser that aren't tightly-coupled to the data being queried (otherwise, they'd be called 'reports' )   

       So I'm very excited by this idea, as it's one of the first times I've seen this type of thing talked about outside of my department at work. There are I think quite a few isolated pockets of people working on this problem though, and already people like the BBC, the UK and US governments and many others have all invested quite heavily in RDF/Linked-Data as a back- end protocol for knowledge-management. There are front-ends out there for visualisation, and things like Gruff do a decent job, but there's a long way to go.   

       //employ a lot of budget creating hand-waving// hand-waving is one of my specialities, when this coincides with budget-creation scenarios, then that's a bonus.
zen_tom, Oct 23 2017

       I think the obvious way forward is not to visualise it, but to auralise it. Hear the data relationships. It might involve employing bats on the payroll, but they're not expensive. It might also explain jazz. Data relationships could therefore formally be encapsulated as dance.
Ian Tindale, Oct 24 2017

       //It might also explain jazz.//   

       Pretty sure that's midi tracks driven by pink noise.   

       //Data relationships could therefore formally be encapsulated as dance.//   

       Pretty sure that's epilepsy driven by pink noise.
bigsleep, Oct 25 2017


back: main index

business  computer  culture  fashion  food  halfbakery  home  other  product  public  science  sport  vehicle