Welcome to the improved data.gov.uk. We have made some considerable changes to the interface on the site, mainly to facilitate discovery, make the site responsive to mobile devices such as tablets and phones and provide hopefully a more elegant experience.
What’s new?
The menu has been redesigned to highlight the key areas our visitors tend to frequent, data being of course the key one.
We have four top options covering the home page, the data catalogue, our apps showcase and the interaction spaces. Clicking on any of them except ‘home’ will provide you with a sub menu where you choose your destination, only the ‘home’ item will take you to a page, the other top items server as a container for destinations, only clicking on one of the sub items will actually navigate away from where you are on the site to your choice.
The interaction space takes over the old participate section of the site and groups those aspects of the site geared towards engagement and conversation such as forums, our documents and guidance library and blogs, as well as some key thematic sections such as linked data and Location. As we add new thematic sections they will appear here.
On the top right corner is the login/logout/profile or ‘user’ button and a data search box, you can now search for data from anywhere on the site. Each area under ‘interact’ has its own search now so content search is area based, not side wide. Only searching for data, our primary purpose, is available across the site. If you want to logout just click on the ‘user’ button.
There is now a small ‘breadcrumb’ line under the top menu which should help you find your way back. It is a beta aspect of the new design and we are still working on it but it can be handy.
The new search page
This is one of the areas where we have made big changes, hopefully for the better (we like to think so). We have provided you with a secondary search bard for data besides the top one, as it may be easier to just click on the right to change your search terms than go all the way to the top. You can also use the map search service from this right-hand side search.
New search facets
On the right-hand side you can refine your search results in very powerful ways. We have three new facets for you. The first one allows you to refine your search result by showing only datasets that are publicly available or datasets that have not yet been made public but are held by departments (unpublished)
You can also show only those datasets that are part of the National Information Infrastructure or hide those to see all datasets not part of it.
The third new facet is themes, currently shown as a beta. We have taken our basic thematic structure, which all publishers have to follow when uploading to data.gov.uk and allowed you to filter your results by each of them. Themes is an area that we are still exploring and this is just our first dash at it.
The results side of the search page is pretty much self-explanatory, you can see the dataset title, the publisher, as brief description, the theme it falls under when provided, file formats available, if the dataset is part of the NII and in the case of maps, the ability to preview it or add it to the preview basket so you can aggregate with others to view.
The dataset page
Once you select a dataset to view, you will be able to see the usual details such as title, publisher, license, 5 star rating, theme and description. Each resource is shown in its own badge and clicking on the right side of the badge gives you the option to preview the file, download it or download a cache version (on this option a tooltip message when moving the mouse over it will tell you the cache date).
On the top right of the dataset page you will find the action icons, which allow you from left to right, to check contact details for the publisher of the dataset, obtain API/Json details for dataset and to see revision version for the dataset.
The publisher pages
Clicking on a publisher’s name anywhere on the site will take you to that publisher’s landing page on data.gov.uk. This page provides information on the publisher and a list of their most popular datasets (based on site views), it also allows you to search this publisher and/or all of its sub-publishers (such as agencies and other bodies)
It also provides you with some powerful new tools accessible from the top right icon action menu.
The first icon is the familiar contact details function. The second item provides you with a view of the publisher and its hierarchy.
The third one is the reporting tool, providing several new reports on each department’s publication commitments, broken links on data.gov.uk and others. The last icon allows you to trigger an RSS subscription to that publisher’s data on data.gov.uk.
That covers the main changes we have made to the site. Some of the new options are in beta, so please bear with us while we continue to improve them.
27 comments
Comment by Anonymous posted on
According to the stats., by riding my motorcycle I`m 57 times more likely to be killed or injured than a car driver and yet I`m penalised if I use a bus lane. I`m constantly filtering between lanes of slow moving cars and can`t use the empty bus lanes in my area.
Comment by Anonymous posted on
great point there on the bus lanes. should definitely be a permanent thing for motorcycles all over the uk.
Comment by Anonymous posted on
I agree with the comment about use of bus lanes, given the raised likelihood of death or injury. I also ride a bicycle - cars frequently encroach onto the cycle lanes in my area (routinely all over Greater Manchester), making it both difficult and dangerous to ride. If I go onto the pavement, them I am encroaching on the space of pedestrians. It often feels as though there's no way to ride safely in either place. I do have some doubts as to how well motorcycles and bicycles would mix in the bus lanes, however.
Comment by davidread posted on
The harvesting interface is still in the publisher menu on the data part of the site, but this menu has moved.
Here's how UKLP publishers should find it:
1. Login
2. Click on "Data", then "Datasets"
3. At the top right of the screen, click the blue lock button and select
"Dataset Harvesting" which takes you to http://data.gov.uk/harvest
Comment by Mark_Halliwell posted on
Thank you David.
Comment by exstat posted on
"The menu has been redesigned to highlight the key areas our visitors tend to frequent, data being of course the key one."
Except that there are three top level headings which deal with open spending, spending over £25K and departmental organisation, all of which are also part of "data", unless they have magically acquired some superhero status. No other part of "data" gets a top level heading. Now weren't there some usage statistics a few months back which (I think) showed fairly conclusively that statistical data were easily the most popular. I don't particularly have a problem with the absence of a top level link for statistics (and it might muddy the waters as far as the National Statistics website is concerned) but the presence of no less than three "entrails of government" top level headings seems to betray an obsession with these that I suspect is not shared by many of your users!
On the plus side, it looks nice, but I'm not sure how many (or how few) months it is since the last one. It may be my unfamiliarity with aspects of the old and new sites but I find it hard (and a bit harder post-revamp) to get an overview of activity in forums. I don't really want to have to open them to find out. That puts me off looking, and therefore contributing or responding, which may be why the number of non-spam contributions on the forum has fallen away so far.
As a final not particularly informed comment, if I type in my postcode, or draw a small area around where I live, I get a listing of about 800 datasets, including (for example) stuff about sparrow hawks, which as far as I know have never been spotted hovering over my local tube station. 550 of the datasets are geographical and only 2 other have themes attached, so not much help in sorting the wheat from the chaff. I must be missing something but how is this facility actually supposed to be of help?
Comment by AAcuna posted on
Exstat,
Good points, let me try and explain a bit further our logic.
The data tab highlights the catalog, which is our primary purpose, Map search, because of the importance to us of the INSPIRE directive datasets and because it is another data discovery tool, the ability to request data, again, firmly in the realm of 'data' as an overarching theme, the publisher's search tool, another discovery tool for datasets through publisher's pages, the organograms tool which is both a visualisation of key government data and a gateway to the linked data API search functions for roles and salaries across government, two reporting tools on specific data - spending data' and analytic on the usage of the site, mainly of datasets.
We feel that this sub options are a comprehensive package of tools to navigate the catalog and to quickly understand some of the key accountability data published by government, we want to highlight them and we feel they are key, although many other logical arrangements would have worked as well, this one achieves what we want to communicate as part of the transparency work being done by the government.
I take your point on the forums completely and we are working on ways to better handle the content in that area, at the moment, it is exactly what it was before and follows the standard presentation of forums listing across most websites, I don't find it ideal and I want to find better ways to pull that information out, but of all the things, front end and back end that we have worked on for this release, forums wasn't on the top of the lists given resources and time. We have to find a better way and we will. I would love to hear what type of interface structure for forums you think would work and get you more engaged.
On the map search. At the moment the map search is part of the work we have done with the INSPIRE directive requirements and UK Location. It only searches across existing map resources (with over 1000 of those on the site) instead of identifying location on files and placing them on a map. The one current shortcoming of the search, due to the nature of the maps, is that even if you specify a post code, if there is a map that contains that postcode it will be pulled, even when the data may not be limited to that area (i.e. the map is a national resource and hence that postcode falls under it even if no data is specifically available for it) . The tool is specifically made to fulfill the requirements of the INSPIRE directive at the moment. We are working with Ordnance Survey to see how we can start pulling in other datasets into the results, although many datasets published do not have location information and it would be impossible for us to determine if they belong to a specific area. Still lots to figure out and it will require work on improving data quality across the piece and working with publishers to get location information included. We intend to make it better with time.
Comment by Owen Boswarva posted on
[no-lexicon]Antonio, I think you're really avoiding the issue by saying the forum content is "exactly what it was before". The content is user-generated; the responsibility of the Data.gov.uk team is the functionality. You have substantially undermined the usability of the forums by reducing that functionality, as follows:
1. Posting histories have been deleted from all user profiles. That means that users (a) have no record of their previous posts, (b) cannot explore the comments and interests of other users, (c) are not alerted when another user responds to a comment they have posted, and (d) cannot easily monitor individual discussions in which they have a particular interest. (These issues apply equally to user comments on dataset records, data requests, Library items, etc.)
2. New threads and posts are no longer promoted on the home page or elsewhere on the site. The only way to identify new threads is to navigate to the forums and then click into each of the forum categories in turn. Effectively that means new threads and "live" threads will go unnoticed by most site users. (These issues apply equally to new documents added in the Library.)
3. Buttons for notification of abusive posts and spam have been removed. This decision is difficult to fathom given the ongoing problems Data.gov.uk has had with spam; as you will recall, last year you let the problem get completely out of hand and ended up nuking a substantial number of threads (legitimate posts and all). There are currently 20 spam posts in this thread alone: http://data.gov.uk/forum/general-discussion/bcs-seminar-linked-data-and-geospatial-semantics
4. The new commenting system itself is inferior to the previous system, as it no longer allows rich text or attachment of links to text. (That's right, the Data.gov.uk forums do not even support proper hyperlinking.)
It's implausible to claim that the Data.gov.uk forums follow the "standard presentation of forums listing across most websites". On the contrary, they no longer provide even the basic functionality users normally expect to find in web-based forums.
-- Owen Boswarva, 02/11/2013[/no-lexicon]
Comment by exstat posted on
Thanks to Owen for setting out the forum-related issues much better than I did. It does seem to me to be fairly basic that if you want a lively forum you make recent activity on it highly visible. It isn't, so ... it isn't. I would however say that some other fora that I use (e g Yahoo Groups) have become much more confusing to use. I think it is to make them easier to use from mobile devices, but the outcome is that it is much harder than hitherto to view a whole thread, and sometimes even impossible to view an individual posting, though it is still easy to see where there has been recent activity. If that represents the new standard, it is definitely not one to be aspired to.
I hear what you say, to use the hackneyed phrase, about the top level headings reflecting the importance that you attach to the transparency agenda. I am sure that Ministers regard it as paramount, and therefore officials have to do so too (a slightly different situation from National Statistics, where the statisticians have a commitment to act independently). But do users? I'm sure it would be good for me if I examined carefully spending over £25K by such-and-such a hospital trust (so would going to the gym, but I don't!). In practice, this particular user finds the list of such datasets postively tedious and - more to the point - something that tends to obscure what I would find of interest. Of course, I may be atypical, but how many people actually do access these datasets? I'm not saying they shouldn't be there, or that the number of users is necessarily a measure of a dataset's importance, but I do wonder whether playing up the transparency angle so prominently actually puts off some who would be interested in the wider agends. And (to return to my first paragraph) contributes to the near moribund state of the fora.
Thanks for explaining the limitations of Map Search. However, perhaps they need to be visible much more up front, to avoid a disappointing user experience. At which point I will sign off before descending further into business jargon...
Comment by Adrian Marsden posted on
OK, so for one day after you went live, the links from the home page to this entry failed.
Also, for us publishers we've lost the links to the harvester settings.
Plus you still seem to be anonymous on your contact page - just a web form. No address, phone number or anything. - Please read the comments on the LGA K-Hub and you'll see loads of people with comments about submitting the web form to get set up and never hearing back.
Deckchairs and Titanic spring to mind. Sorry.
Comment by AAcuna posted on
Adrian,
I am sorry you feel like that. The links on the home page were set to a self-referal loop until the prime minister spoke at mid-morning on the Thursday. A few were not added until later on Friday as the team was helping with the running of the Open Government Partnership conference in London.
I would hope that people will end up judging the site by the totality of the effort especially how much easier it is to navigate and interact with as well as how much better administration by publishers is.
As you probably saw from the answer by David below, the harvesting link is there.
Not clear on your comments relating to the contact page.
I actually think the ship is sailing very well, wobbles in rough sea, certainly, but definitely solidly moving onwards. It is a beta site developed and managed by an amazingly small team and I believe the results are outstanding, especially compared to the functionality available by other government data portals around the world (I refer to functionality and layout) We have issues, sure, but I am perfectly comfortable with the minor scale of those issues compared to the end result (and we will fix those issues)
We are actually working at the moment with the LGA to set up a proper framework for Local Authorities publication, that should help with some of the issues encountered with registering publishers.
Hopefully you'll have a chance to benefit from the new design.
Cheers,
AA
Comment by Mark_Halliwell posted on
As the data publisher for the UK Hydrographic Office, I have just logged on today in order to initiate the harvesting of 1400 new datasets. I am very surprised to see that the harvesting facility has been removed. This appears to be a deliberate change of policy, rather than an omission, as the new guidance for publishers no longer mentions this option.
I fully accept that for many publishers it may be practical to upload one dataset at a time, but for us it is not. The datasets in question are bathymetric surveys from around the coast of the whole of the UK (ie part of the 'Elevation' INSPIRE theme).
There appear to be two ways forward: either the harvesting functionality is reinstated, or I create one metadata record which describes the superset of all 1400 datasets. This would dramatically reduce the usefulness of the geographic search facility on data.gov.uk
Comment by Adrian Marsden posted on
Antonio
You didn't address the issue of contacts. For this web site not to have contact details of real people is shocking. There are people on the K Hub who have been waiting months to be set up as a publisher. Without real people to contact there seems to be no way to progress these once the poor web form has been filled in.
I'm lucky, I now have a real person email address I use when I have issues, but as someone who does 2nd & 3rd tier support, I get annoyed when users contact me directly, I understand that this is not ideal and I use it carefully.
You must be aware that, in theory, we have a deadline of only a few months now to get our INSPIRE data on. So to keep people waiting weeks and even months before they get set up just isn't acceptable.
ACM
Comment by AAcuna posted on
Adrian,
I take your point and I will look now into adding another contact channel for those requesting publishing rights. Not aceptable, I agree. I will work with the Location helpdesk to make sure everyone has a couple of contact details they can use just in case.
Cheers,
AA
Comment by phtr posted on
Hi - I'm not sure if this is the correct place to raise this but it seems to me there is a minor error on the dataset/service details pages. If a dataset has coupled services, the dataset page has an entry in the 'additional information' section called "Coupled Datasets"; similarly services have an entry called "Coupled Services". I think these labels are the wrong way 'round, ie:
Comment by Alienor posted on
Looking back at this blog today, and trying to guide a colleague around it and the site in general I was suddenly struck by the inconsistency in date formats. This is confusing. You are using the American format here - month first and yet for the data you use day then month. Is there any reason why we are not using the international standard date notation YYYY-MM-DD ?
I would also like to add my support to the comments already made about the forum format. It is frustrating being unable to easily identify new threads. I wonder if the new site design has also made it more difficult for people to find as there seem to be hardly any new comments on the forum under any subject.
Comment by AAcuna posted on
Alienor,
Thanks for your comments. I am a bit confused as to your 'month first' comment. The blog timestamps are day/month/year so it is for the comments, the date fields for publishing data and the dataset date display (unless the publisher ignored the format required and typed the date, which we know happens and are trying to educate publishers about this)
YYYY-MM-DD is not the UK date format, which is DD/MM/YYYY this format is required, for example, by treasury for the reporting of government expenditure.
Can you point me to an example where we may have used month first format so we can correct it?
As for the forum, we are in the process of looking for a better forum module for the site, as the current one does not provide the flexibility we require. We'll have more on that soon once we have a working version.
AA
Comment by Owen Boswarva posted on
Antonio,
I can confirm there was an issue with date formats on the site, as described by Alienor.
Here is a Google cache snapshot from last week of a Data.gov.uk page with a system-generated date in the US-style MM/DD/YYYY format:
http://www.freezepage.com/1384553585ZFVUGLFHUG ("Submitted by Lizetta on Wed, 10/30/2013 - 18:26")
Offhand I cannot find any live examples of this issue, so assume it must have been remedied recently by one of your team.
Good to hear the forum is still being looked at.
-- Owen Boswarva, 15/11/2013
Comment by AAcuna posted on
Owen,
Thanks for that, interesting, we may have had a glitch!
We have not forgotten the forums, not at all. We are looking for a decent forum infrastructure that gives us the functional framework so we can move away from forums being in a static corner and also play a part across the site by being related to topics, datasets and blogs, that way we can pull them in when pertinent across the site and add more contextualisation between dataset/conversations/blogs and library. Resources are very tight right now and that is why we haven't been able to go back to the existing solution and do further work.
Just give us a bit of time to get it right and we will provide a better experience.
Cheers,
AA
Comment by simonliu posted on
We discuss about data.gov.uk recently. The data site keep on the CKAN's dataset but has own data previewer. Why not CKAN's dataset move to drupal site using CKAN's RESTful api or api? That is, move CKAN site to internal site. Drupal site get data, or search data via CKAN api. The UK's website all use drupal and manager by drupal.
Or, is this UK next step?
-- Simon, 19/11/2013
Comment by AAcuna posted on
Simon,
Actually, using Ckan for both cataloguing and interface has been the best move we could have made. Drupal is a great and powerful CMS, but not necessarily a bespoke data cataloguing system. There are good products, such as Dkan, but we are firmly set on Ckan as the core catalogue/interface for data.gov.uk.
Over 10 countries have also followed that direction and many made the switch from a Drupal centric solution to Ckan catalogue/interface for the same reasons: flexibility, simplicity and a very powerful python based environment.
We have no plans to revert this trend nor do I find the interface capabilities of Ckan lacking, quite the opposite, I find it far way more nimble, responsive and flexible than what we had with the Drupal front end only site.
Our experience shows that although Drupal has an amazing community behind it, it still presents quite a resource overhead to get it pass the out-of-the-box experience. Also, the out-of-the-box experience is simply not well thought out and lacking sometimes some of the most basic of functionalities already found in CMS software in the last 10 years, this has been repeatedly mentioned in many of the Drupal forums by seasoned Drupal developers, maybe 8 will bring a better experience.
You will also find that for example, Gov.uk, the government website, uses a mixture of solutions. http://digital.cabinetoffice.gov.uk/colophon-beta/
Cheers,
Comment by tonyrogerson posted on
Do you have a timeframe to when we can use ODATA to access datasets?
Also, the search doesn't appear to be working correctly - I searched ODATA but no results, but through google I got to a forum post.
Many thanks,
Tony
Comment by AAcuna posted on
Tony,
Good point. We are working with Microsoft, who has agreed to cooperate with us to improve interoperability with Odata. We are working on implementing the standard and should have something working in the first part of the new year.
I'll make a note of the search issue you encountered and see why it happened.
Cheers,
AA
Comment by chaz6 posted on
Has there been any progress on this? I would like to see odata services being published for consumption.
Comment by Keith posted on
Having to use IE 8 and the format of the site is now lousy, with lots of right hand panes overlaying the central text. As a result the opening line for this page now reads "Welcome to the improved data.gov.uk. We have made some considerable c"
Home page also looks awful. With "National action plan" in black over the text "Open Government Partnership" in white.
Can't change the browser as we are using National systems that are incompatible with later versions of IE.
Comment by AAcuna posted on
Keith,
Unfortunately, we cannot support older versions of browsers going back to 2009 (when IE8 was released) I would suggest that you upgrade to IE9 or IE10 or use the latest builds of Chrome, Firefox or Opera to experience data.gov.uk (and many other sites) properly.
If you are using a corporate machine and have no ability to change or upgrade browsers, one possibility is that you request that your IT department install Chrome frame in IE8, which will allow it to display pages properly, although Chrome frame is to cease development this year, older versions could still provide you with the functionality IE8 lacks.
In order to provide the best experience and be able to interact with modern server and page display technologies we only support current browser versions.
Regards,
AA
Comment by exstat posted on
What does "current" mean? Clearly not "latest" or IE9 would not be an option. You don't mention IE11, curiously, which I am using. There is a certain irony that a site that is about making data available in a form that can be used my many forms (and versions?) of software cannot be viewed effectively by browsers that are still in everyday use. That can't go on for ever, of course, not least because there comes a point when development is stultified. But I'm less fussed about getting "the best" rather than a good experience.
Government was, maybe still is, the worst offender. In my latter days at the department which promotes innovation and technology (BIS) we were stuck with IE6 (I think) well beyond its sell by date and regularly used to get messages about it from websites offended by our backwardness. I think we even got such messages from our own website sometimes. We were told that it was a major task to check that nothing else would be messed up by interaction with a new version. It is also possible that the contracts with IT providers are drawn up in such a way that the cost to the client becomes exorbitant.