What is Open Data?
Open data is the idea that public information should be accessible, used, modified and shared by anyone for any purpose without restrictions. Open data should be both technically and legally open: technically open in that it is available and accessible to the community, and legally open in so far as there are appropriate licensing frameworks in place to facilitate its release, use and reuse by the community.
What are the benefits of Open Data?
The NSW Government's open data policy is designed to drive benefits to the community because Open Data:
- Promotes open, transparent and accountable government;
- Contributes to the digital economy in NSW, and promotes the development of new business and industries that can make use of government data;
- Leads to better public services;
- Advances community engagement with the government and with the work of government;
- Enables data sharing between government agencies in NSW and across jurisdictions;
- Facilitates greater understanding of agencies own data and the potential of that data; and
- Supports evidence-based policy making and policy research.
What is a dataset?
A dataset is a collection of related sets of information, which shows relationships between categories and variables. An example of a dataset is a table of all train stations in NSW, with a set of information about each station, such as platform numbers.
What is the difference between a dataset and a resource?
Datasets contain individual resources, as well as metadata. Metadata contains information about the dataset. Resources are the files within the dataset. A dataset may have multiple resources such as APIs, links or images.
What are APIs?
Application Programming Interfaces (APIs) are a set of rules and instructions for how software can interact with a dataset. APIs allow flexibility in how information within a dataset is accessed, typically so that users can select only a specific, relevant subset of the information. APIs are usually connected to datasets that are updated in real-time.
What type of data are TfNSW offering?
TfNSW offers a number of datasets and APIs for all public transport modes and roads. These include static timetable information, as well as live and real time information (feed data). If you have an idea of data that you think would be useful, let us know on the Forum.
The Open Data Program
Why is Transport for NSW opening its data?
Making more government data available to the community is a key commitment of the NSW Government, is a priority of the NSW Government ICT Strategy, and is a driver of the NSW digital economy. By providing data to the community, we can improve transparency and accountability, enable evidence-based policy decisions and provide a platform for innovation. TfNSW has provided open data since the launch of the TDX Program in 2009. TfNSW is committed to fostering innovation in the digital economy, and is excited to see the innovations and creations the community produce using our data.
Does the TDX program still exist?
The Open Data Program replaced the TDX Program in April 2016. The Open Data Program is providing access to a broader range of information than the TDX Program, including the provision of APIs.
Is Transport for NSW the only NSW Government agency opening its data?
The NSW Government has made a commitment to opening more data to the public and many NSW Government agencies have begun releasing open data. Examples include Family and Community Services, Property NSW and the Information and Privacy Commission. For a full list of NSW Government agencies providing data to the public, please visit data.nsw.gov.au Open data is not only a NSW initiative and is quickly becoming a priority for governments across the world, including Europe, North America and across Australia too! To find out more about our federal open data please visit data.gov.au.
What is the Open Data Hub and Developer Portal?
The Open Data Hub and Developer Portal is where you will find all publicly available TfNSW open data. It provides access to all APIs and datasets, as well as developer information to get you started and a Forum to contact us.
Who owns the Transport for NSW open data?
Most of the data published on the Open Data Hub is provided by Transport for NSW or other associated entities. Ownership is usually indicated for every dataset.
Using Transport for NSW data
What do I need to know before I start using TfNSW data?
What can I do with open data?
The possibilities for using TfNSW data are endless and already we have seen many innovative ideas! The data can be used to produce maps and reports, or used by policy makers and transport planners to gain insights, for analysis and informed decision making. Most commonly, the data is used to create transport apps, both for public transport modes and roads. You can check out some of the apps developed using TfNSW data in our showcase.
Is TfNSW data accurate?
We derive and directly source our data from operational systems within TfNSW. While we cannot guarantee the complete accuracy of the data, it is the best source of data for our transport services.
Is new data added to the Open Data Hub and Developer Portal?
We are constantly working to release new datasets and APIs. You can request data by emailing OpenDataProgram@transport.nsw.gov.au.
How is data containing private information protected?
TfNSW sets out to provide as much data as possible, and some datasets need to be cleansed before being released. This may be due to safety, commercial concerns or protecting the privacy of our customers. These datasets need to be identified, prioritised and made available in the appropriate format.
Can I host my own data or app on the Open Data Hub and Developer Portal?
No, the Open Data Hub and Developer Portal is designed to provide data for you to use and is not a hosting service.
Can I publish my own data to the Open Data Hub and Developer Portal?
Currently, only TfNSW can publish data to the Open Data Hub and Developer Portal.
Can I use my own API?
You can use your own API, but you will need to host your own API or use another host.
Do I need to have technical knowledge to access the Open Data Hub and Developer Portal?
The Open Data Hub and Developer Portal is accessible to anyone. While access to APIs is designed specifically for developers with technical knowledge, we also offer the ability to download datasets which requires no technical knowledge. Downloadable datasets are often used by researchers who are interested in TfNSW data. If you get stuck, the Forum is available to everyone. Through the Forum the community and TfNSW staff can assist with technical queries and general questions about the data.
What data formats are used?
Static type data is available in text formats, usually in a zip file. Feed data is available via APIs in specific formats. Check the Data Catalogue for more information.
Where are the code samples for the API Explorer?
The new Open Data Hub and Developer Portal, does not display code samples. A cURL command is provided to test your API details.
I’m using TfNSW data to create a product; is there any design or branding guidelines I need to follow?
The use of TfNSW data should not be seen as an endorsement, sponsorship and/or partnership and you should never brand your product in a way that states or implies this. Visit the Marketing section of our Resources page for full details of what you can and cannot use to brand your product.
Can I reuse or redistribute TfNSW data?
For any dataset or API with a Creative Commons by Attribution 4.0 licence, you may reuse and redistribute the data. Please check the licence for the data you are using.
How can I share Application credentials?
The creator of the application can share the API credentials (API key or OAuth credentials) with other users manually. There is no system functionality enabled to share apps or API credentials. **Please note:** you should never share your user credentials (username and password) with anyone.
How can I have my product featured on the Showcase?
You can request to have your product included on our Showcase by emailing OpenDataProgram@transport.nsw.gov.au with the following information: * An overview of your app or website * Features or services you offer * A high resolution logo * Screenshots of your app or website.
Why isn’t the API Explorer working for the Timetables Complete GTFS and TransXChange APIs?
The Timetables Complete GTFS and TransXChange APIs return large files, and is unlikely to work in the API Explorer (or when Explore API is selected in the Data Catalogue). You can download the zip files by selecting Download in the dataset. Should you wish to retrieve the files on a regular basis, you can use your API key to retrieve it programmatically.
What methods of authentication can I use for my apps?
We recommend using API Key authentication as it is simple and has fewer issues than the OAuth 2 authentication. If you are using OAuth 2 authentication you may continue to do so.
Why are there multiple records for the same ferry in the vehicle position feeds?
The GTFS-realtime ferry feed reports all current and future vehicle positions. This causes a "trail" of vehicles to appear while the ferry is moving. It's possible to filter out to only show the current trip's vehicle position. Use vehicle > id as the unique identifier. Since the ferry feed no longer reports vehicle positions once a trip has been completed, you can cross reference with the GTFS-static data to identify which trip begins the earliest (i.e. has the earlieststop_times.txt entry).
How often are real-time data feeds updated?
Sydney Trains GTFS and GTFS-R: Real time data is updated every 15 seconds, static data is updated once a day
Bus GTFS and GTFS-R: Real time data is updated every 15 seconds, static data is updated once a day
Ferries GTFS and GTFS-R: Real time data is updated every 15 seconds, static data is updated once a day
Light Rail GTFS and GTFS-R: Real time data is updated every 15 seconds, static data is updated once a day
Rural and Regional GTFS and GTFS-R: Real time data is updated every 15 seconds, static data is updated once a day
How do I get access to Open Data?
To access TfNSW data you will need to register and sign in via the Open Data Hub and Developer Portal. The register button can be found at the top of the page.
How can I search for the data I need?
All available datasets and APIs can be found in the Data Catalogue. If you can’t find what you are looking for, you can post a question in the Forum. Be sure to read previous posts before posting a question, someone else may have already asked.
The data I need isn’t available in the Data Catalogue, how do I request it?
You can request additional data by emailing OpenDataProgram@transport.nsw.gov.au.
Where can I find information about terms referenced in the data and datasets?
The Documentation page contains lots of information to help you understand our data.
How do I ask questions about the data?
You can post questions in the Forum, where the community and TfNSW staff can assist with your query. Be sure to read previous posts before posting a question, the answer may already be there.
Can I just browse the data?
You can search for all our available APIs and datasets in the Data Catalogue on the Open Data Hub.
How can I request data?
You can request additional data by emailing OpenDataProgram@transport.nsw.gov.au. For more information, go to the Data Request page.
How long will it take to receive requested data?
Timeframes vary based on the data being requested and whether it is already available on the Open Data Hub, requires additional investigation or can even be provided. We do endeavour to get back to you when you make a request, so if you haven’t heard from us please let us know as your request may have gone on a detour.
Can I request non-transport data?
We only provide Transport for NSW related data.
How can I find out what your latest data releases are?
Data releases are announced on the Open Data Forum as well as through our Twitter account, so keep an eye out there for the exact date. You can also sign up to our mailing list to receive our quarterly newsletter and other updates including Innovation Challenges.
Are CCTV feeds available?
Due to privacy protection policies and legal restrictions, we are unable to provide or release any CCTV footage.
What is an Innovation Challenge?
At Transport for NSW (TfNSW) we use innovation challenges as an efficient way to develop and deliver products and solutions that improve the customer experience across Sydney and NSW. The Open Data and Innovation (ODI) team runs innovation challenges periodically to seek out those best ideas from industry players to bring better outcomes for customers. Innovation challenges are run when TfNSW has a good understanding of the potential solution or product which will solve a problem, and where we want to work with the startup community to develop a solution. View upcoming and past challenges on our Innovation Challenges page.
How do our Innovation Challenges work?
The ODI team runs innovation challenges with a variety of focuses based on different business needs within the Transport for NSW cluster. It enables a path for developers to have their products officially endorsed by TfNSW. Innovators, entrepreneurs, and the developer community are encouraged to participate in the challenges to address the problem statement(s) with their proposed ideas and solutions.
What is the App Developer Program?
Solutions that successfully pass incubation with Transport for NSW (TfNSW) become part of the TfNSW Endorsed App Program. The App Developer Program provides support to TfNSW endorsed app developers. Endorsed apps have come through innovation challenges held by TfNSW and are managed by the Open Data and Innovation (ODI) team. Innovation challenges are held throughout the year, with various focuses to meet TfNSW’s overall strategy and to solve real transport problems.
Help and support
What is Odie bot?
Odie bot is a chatbot service where you can ask questions about Open Data Hub. It can also help you find data or answer any questions about using our API.
Where can I get help using the Open Data Hub and Developer Portal?
We have a number of channels to help you: If you need help with data or technical questions, please read the Getting Started and Documentation pages. If you still have questions, you can post a question in the Forum. Be sure to read previous posts before posting a question, the answer may already be there. If you have lost your password or username, or need to report an outage or issue, send an email to OpenDataHelp@transport.nsw.gov.au. We aim to respond to requests within 3 – 5 business days. Be sure to add as much information as you can to help us get back to you more quickly.
How do I report data issues or outages?
Data issues and outages are shown on the Status page. Check the Status page first to see if it has already been reported. If the issue or outage isn’t displayed on the Status page, email OpenDataHelp@transport.nsw.gov.au with details of the issue or outage. Please be aware that some issues and outages cannot be fixed quickly, and may take time to resolve. You should always report issues and outages as soon as possible, so we can investigate and let the relevant people know.
How can I stay up-to-date with TfNSW open data?
The best way to stay in touch with the Open Data Program is to follow us on Twitter @datatfnsw for regular updates and visit the Forum for more information about our data and more!
How can I receive dataset update notifications?
The Data Update Notification Service (DUNS) allows Open Data Users to subscribe to update notifications for their chosen datasets. Simply log in to the Open Data Hub, navigate to the dataset you are interested in, and click "FOLLOW". You will receive an email to your nominated email account notifying you if the selected dataset has changed in the last 24 hours.
Am I able to change the time of the email notifications for the dataset updates?
Unfortunately, there is no way to change the time for the email notifications. The email will be sent out at 00:00 GMT.
Where can I see the datasets that were changed in the last 1 month?
You can visit the “Data Updates” page by clicking here. There is a filter where you can filter the type of changes which you are interested in.
How do I use GTFS Studio?
You can access GTFS Studio from the top menu items on the Transport for NSW Open Data Hub. You will need to be logged into your Open Data Hub account to access and must be using a desktop resolution. You can review the user guide by visiting the Documentation page.
What is GTFS?
GTFS stands for General Transit Feeds Specification. More information about GTFS can be found via https://gtfs.org/. It’s the key format of providing public transport data to developers and planners for timetable and realtime data.
What’s the best way to view GTFS data?
GTFS Studio is a feature added to the Open Data Hub as part of the Make Data Great Innovation Challenge. It provides access to browse Transport for NSW GTFS data and GTFS-realtime data.
Otherwise GTFS data can be downloaded as a zip file and are CSV files which can be viewed in a text editor or Excel or spreadsheet application. Realtime data needs to be converted as it is in a protobuf format - our suggestion is to use GTFS Studio if you want to have a look at the data.