Troubleshooting

Tips and tricks for troubleshooting your applications. 

The data we provide on the Open Data Hub is often acquired directly from an operational system, which means a lot of work happens in the background to cleanse and make the data more useful. However, there are still a lot of quirks when working with our data. We have learned a lot from third-party app developers and have put together some tips and tricks to help you troubleshoot your problems.

HTTP standards

I’m having trouble checking whether an API has been updated or not

Some APIs return a last-modified header and timestamp in the HTTP response. If you send this timestamp in an if-modified-since header, you may not get back an HTTP status of 304 (not modified). Instead, in some cases, you may need to use the HEAD HTTP method to retrieve the last-modified timestamp without retrieving the data.

I’ve encountered some issues with OAuth 2 authentication

The OAuth 2 authentication method was the old way of connecting to our APIs in the previous version of the gateway/portal. OAuth 2 allowed a non-standard security protocol for obtaining OAuth tokens. We now recommend that you only use the API key method of authentication, which is simpler and safer to use.

I’m getting a HTTP 401 error when calling an API, what’s wrong?

If you’re getting a 401 error when you call an API it means that you’re either not passing your authentication parameters correctly or you have not selected the API you’re using in your application setup. For more information on how to set up your application and use our data please follow our User Guide.

Real-time Public Transport (GTFS-R feed)

Why can’t I find all services in the GTFS-R data feed?

The GTFS-Realtime data feed does not include any scheduled services that are not real-time. To see all scheduled services you need to look at the Timetables Complete GTFS (GTFS bundle) dataset or the TXC bundle. Also make sure to have a look at the Overlapping Agencies in GTFS Feeds list that outlines all the agencies with real-time services.

How do I ensure customers have the most accurate information in my app?

Real-time data isn’t always accurate. Various issues such as connectivity and GPS issues can affect the data feed. If you are developing a real-time app always make sure to use scheduled data as a fall back indicate when data is real-time as clearly as possible.

The API response is in a weird binary format that is unreadable, what do I do?

When you call the API, the response will be returned in binary format. You need to use a .proto file to convert the data into readable format and apply our modifications. To learn more about protocol buffer files go to https://developers.google.com/protocol-buffers/docs/proto

I’m calling the real-time APIs every second and hitting my quota limit, what can I do?

The real-time files are updated every 10 to 15 seconds from the operational systems therefore there is no point in calling the APIs every second. We recommend that you call the APIs and update your app data every 15 seconds.

I’m having trouble identifying buses with special livery or Wi-Fi

The buses real-time feed (Vehicle Positions API) has a special_vehicle_attributes parameter that signifies whether a bus is a 85th anniversary bus, has Wi-Fi or is a Christmas bus depending on its value. For more information please read the buses technical documentation.

I’m including future projections in my app, why am I seeing different ranges of future dates between modes?

Future projections for data feeds might not necessarily match between GTFS and real-time since they come from different systems. Some systems require the feeds to have at least 100 days of data in the future while others have a set number of days. We recommend that you always use the latest available bundles and files to ensure that the data is up to date and reliable.

Public Transport Service Alerts

I’m having trouble setting up service alerts in my app

Alerts are available for all modes in GTFS-R format. You can also use the Trip Planner API add_info endpoint to get full service alerts for all modes. This will make your app more efficient and prevent the need to clean up duplicate entries if using multiple feeds. For more information about this please make sure to read the Trip Planner API manual.

GTFS and GTFS-R Updates

I’m not sure when and how often I should update my app’s GTFS files

The GTFS files will update daily overnight with each transport mode having a different time. Therefore you should only need to download/update your files once per day. For more details about each mode see mode specific sections below.

I’m not sure how often I should update my app’s real-time data

The real-time files/APIs are updated once every 10 to 15 seconds, therefore we recommend you update real-time data every 15 seconds. There is no point calling the APIs more frequently than this since the data won’t change and you risk hitting your quota or rate limit.

Sydney Trains

When does the Sydney Trains GTFS bundle update?

The Sydney Trains GTFS files update daily at around 02:00am. Updates will contain regular changes of trip/calendar keys while in most cases shapes can change without the shape_id changing.

What are RTTA_DEF and RTTA_REV services in the real-time feed?

Services with route_id as RTTA_DEF or RTTA_DEF are non-revenue services and should be excluded by app developers and not presented to customers. Sydney Trains Charter services should also not be displayed to paying customers. Check the Sydney Trains release notes for the relevant run numbers.

I’m seeing duplicate services in the Trains feed, what am I doing wrong?

You need to exclude some services from the NSW Trains and Sydney Trains feeds to avoid duplication, for more information about this please make sure to read the NSW Trains technical documentation.

There seems to be additional stops in some trips that aren’t used

Extra stops are presented in the Sydney Trains feed as they include passing stops in trips that are flagged as no set down/no pick up, and should not be passed on to customers. The stops are not included at all in the Complete GTFS dataset, as they effectively aren’t in use.

I’m having trouble producing the right shapes for train lines

Best practice is to have a look at the shapes.txt file included in the GTFS bundle and use the lat/long coordinates to produce your own shapes. Be aware that shapes do not always match trips 100%. There may be missing stops at the beginning of the shape or extra stops at the end of the shape.

I’m having trouble working out train delay information

To determine the train delay information for scheduled services (ie. have a trip schedule_relationship of SCHEDULED), only a delay field is provided. You should obtain the scheduled arrival time and departure time from stop_times.txt (in the GTFS for real-time dataset) and add the delay seconds. Negative value denotes the vehicle is expected to arrive or depart ahead of schedule. Match each stop_time_update by the stop_id provided in the realtime and static feeds.

What about delay information for replacement services?

To determine the train delay information for replacement services (ie. have a trip schedule_replacement of REPLACEMENT), these include a timestamp for arrival and departure. Delay field is also provided for stops which were already part of the schedule (in stop_times.txt) and have not been skipped, added or replaced.

I’m not sure how much time I should have for transport connections and transfers in my app

To figure out the amount of time needed for connections please have a look at the following documentation, Connections Guide RSDO 2017 Timetable.pdf.

Why am I seeing changing trip and service IDs?

The timetable ID and version ID of each weekday/weekend train plan is used as a component in GTFS service/trip IDs generated by RTTA. Any single change in a train plan will increment the version ID meaning all related trip/service IDs will change.

Why does the start date for the active working timetable keep changing?

The current behaviour in RTTA is that the startDate increments to be at minimum the current day, so for the current active daily working timetable will change daily. The GTFS bundle keying now changes daily at minimum of 5 days a week.

Why am I seeing the REPLACEMENT schedule relationship value? I thought that wasn't part of the spec any more.

The current official GTFS-realtime specification is not compatible with the Sydney Trains GTFS-real-time feed. REPLACEMENT has been removed from the official GTFS-realtime specification but the Sydney Trains feed still contains it.

NSW Trains

When does the NSW Trains GTFS bundle update?

GTFS files for NSW Trains updates daily. The bundle covers the next 35 days of the daily working timetable. Each trip is defined with a unique ID for each day it will operate on, so there is long term key consistency 34 days in advance. For more information about this please make sure to read the NSW Trains Technical Documentation.

Buses

When does the Buses GTFS bundle update?

GTFS files for buses updates daily, normally around 3:30am. Has key consistency over time - the key of a trip appears to be linked to the definition of it, so if the route of a bus changes temporarily then a new trip ID will be assigned to it.

Why am I seeing trips being added and removed from the buses feed?

The buses feed will periodically have new sets of trips added/removed from it as operators upload new data - sometimes only 1-2 days before the added trips will commence operation (so updating regularly is important, but taking a few hours to process updates is fine).

I’m having trouble finding the features of a bus

Features such as air conditioning or wheelchair access can be found in the vehicle descriptor parameters in the Vehicle Positions API. The vehicle descriptor extension is described in the TfNSW Proto File.

I’m having trouble finding the vehicle occupancy indicator for buses

The vehicle occupancy indicator can be found in the Vehicle Positions API for bus services.

I’m having trouble finding what the passenger occupancy status for buses mean

The degree of passenger occupancy of the vehicle can be found in the TfNSW proto file.

Ferries

When does the Sydney Ferries GTFS bundle update?

The GTFS files for Sydney Ferries update daily. Trip IDs are mostly consistent over time.

Why do the calendars for ferries keep changing?

Calendars in the ferries feed can change on a daily basis, particularly around public holidays, due to limitations of the operational system timetabling. Always make sure to use the latest bundles.

Light Rail

When does the Light Rail GTFS bundle update?

The GTFS files for Light Rail services are updated ad-hoc as necessary since it is a manual process. It’s best to monitor it periodically.

Why am I seeing platform code numbers in the feed if Light Rail stations don’t have platform numbers?

Platform_code is populated. Light Rail stops do not have platform number displayed on the ground, but these have been added to indicate a direction for each. 1 = toward Central, 2 = toward Dulwich Hill.

Live Traffic Data

How often does Live Traffic data update?

There are no set intervals for updates, the data feed is updated whenever something changes.

Other Tips, Tricks and Info
  • For the Sydney Trains feed be careful when applying trip_headsign and stop_headsign correctly so “empty trains” do not appear. For app developers who do not know the Sydney Trains network, there is a loop. Please check the GTFS Specifications and apply block_id to join services.
  • There is a character limit on Sydney Trains headsigns so you need to do a transformation.
  • For the Sydney Trains feed, the calendar service_id's are constructed using the format {timetableId}.{timetableVersionId}.{dopRef}-{startDate}, while the trip_id’s are constructed using the format {tripName}.{serviceId}.{setType}.{numberOfCars}.{tpsTripId}.
Need more help?

If you need more help please have a look at our Developer Resources and also make sure to read all of our Technical Documentation before you start working with our data and APIs.