Teaching Alexa to Spot Airplanes

Fun with RTL-SDR and Amazon Echo Dot

June 2017

A few months ago, I moved into a new apartment that has a sweet view of passing planes.

airbus a380

As an airplane fanatic, I’d often go to FlightRadar24.com to look up the different airplanes as they passed by, but after a while that became cumbersome.

So, I taught Alexa to do the work for me! Here’s the end result:



This was actually a pretty straightforward job. The first step was to make two new electronics purchases (not including my Echo Dot):

The dongle is essentially a radio receiver for your computer that can pick up most unencrypted radio broadcasts. These broadcasts can range from police chatter to weather data, but the relevant broadcasts here come in a format called ADS-B, which stands for Automatic Dependent Surveillance–Broadcast. Airplanes continuously send out ADS-B messages to share important info like position, velocity and callsign with air traffic control as well as other planes.

By plugging the dongle into the raspberry pi (and placing by a window with an unobstructed view), and with the help of some open source software, the Pi can be turned into a cheap ADS-B decoder/server!

Receiving Airplane Message Data (ADS-B)

After installing all the necessary drivers for the dongle, I used an open source program called dump1090 to decode and translate the incoming ADS-B messages into easily readable json objects. This makes getting json data about nearby airplanes as easy as running ./dump1090 --net and checking localhost:8080/data.json for the output.



browser local host api call

The {"hex": "abd204"} key-value pair seen at the beginning of the response above is the airplane’s unique ICAO code. With it we can look up the aircraft’s registration number, model and operator. I used the csv data from Jun Zi Sun's Aircraft Database and loaded it into a local Mongo DB collection, which I query using the ICAO hex code every time I scan for nearby planes.

def db_results(icao24):
  client = MongoClient('localhost:27017')
  db = client.AircraftData
  result = db.Registration.find_one({
    'icao': icao24.upper()
  })

  airline = result['operator']
  reg_no = result['regid']
  aircraft = result['type']

  return reg_no, aircraft, airline

In case there's more than one airplane nearby, I use gpxpy to calculate the distance of each plane to my window and then choose the closest.

def distance_from_window(flight):
  window_lat, window_lng = settings.window_coords
  return gpxpy.geo.haversine_distance(window_lat, window_lng,
               flight['lat'], flight['lon'])
        

The only data left to retrieve is the flight’s departure and arrival airports. Using the registration code, this can be scraped from FlightRadar24's data/aircraft/ endpoint using BeautifulSoup (if there's a better way, please share 🙂); however, the scraping itself was slightly more involved than I was used to.

The html response from FlightRadar24 includes a number of past and future flights for a given aircraft, but does not indicate the current flight, so I had to infer the current route by finding the flight with the most recent departure time.

This led to another hiccup: the returned flight times were relative to the departure airport timezone and were still in GMT. To normalize/localize the flight times, I used a mapping of airport code to GMT-offset that I found online which I loaded into Mongo. Here's my final scraping solution:

def get_departure_airport(row):
  airport = row.findAll('td')[2].find('span').text
  airport_code = re.search('[A-Z]{3}', airport).group(0)
  return airport_code

def get_tz_offset(airport_code):
  client = MongoClient('localhost:27017')
  db = client.AircraftData
  result = db.AirportTZ.find_one({
    'code': airport_code
  })
  return abs(result['offset']['dst'])

def departure_time_for_row(tr):
  tds = tr.findAll('td')
  if len(tds) < 6 or tds[6].text.strip() == '-':
    return None
  year_month_day = tds[1].text.strip()
  time_depart = tds[6].text.strip()
  localtime = datetime.datetime.strptime('{} {}'.format(
            year_month_day, time_depart), '%Y-%m-%d %H:%M')
  departure_airport = get_departure_airport(tr)
  return localtime - datetime.timedelta(hours=get_tz_offset(departure_airport))

def std_in_past(row):
  std = departure_time_for_row(row)
  return std and std < datetime.datetime.now()

def most_recent_departure(soup):
  trs = soup.findAll('tr')[1:] # first tr in html isn't a flight row
  return next((tr for tr in trs if std_in_past(tr) and tr is not None), None)

def scrape_route_data(reg_no):
  url = route_data_endpoint.format(reg_no) #flightradar24.com/data/aircraft/{}
  res = requests.get(url)
  route_row = most_recent_departure(BeautifulSoup(res.text, 'lxml'))

  depart = route_row.findAll('td')[2].find('span').text
  depart = re.sub('[A-Z]{3}', '', depart).strip()
  arrive = route_row.findAll('td')[3].find('span').text
  arrive = re.sub('[A-Z]{3}', '', arrive).strip()

  return depart, arrive
            

Finally, I set up a public node js server on my raspeberry pi that pulls the airplane data and formats it into a good ol' English sentence when requested. Then, I simply have Alexa make a request to the pi server when prompted and read back the result.


If you're not familiar with how Alexa skills work, you basically create a new skill on the Alexa Skills Kit dashboard, give that skill a name and various utterances that can invoke it ("what plane is that?") and point it to an AWS Lambda function or similar service capable of processing an Alexa skills request. You don't have to use AWS Lambda, but it's definitely a quick and easy option for simple skills. Mine simply returns a basic Alexa skill response with the formatted airplane data from the pi server.

from __future__ import print_function
from lambda_settings import app_id, speech_endpoint
import requests
import json

def is_valid_app(event):
  return event['session']['application']['applicationId'] == app_id

def get_output_speech():
  r = requests.get(speech_endpoint)  #pi server's public IP
  output = json.loads(r.text)['response'].encode('ascii')
  return output

def get_response():
  return {
    'version': '1.0',
    'response': {
      'outputSpeech': {
        'type': 'PlainText',
        'text': get_output_speech()
      },
      'card': {
        'content': 'Planes rule!',
        'title': 'Plane Info',
        'type': 'Simple'
      },
      'reprompt': {
        'outputSpeech': {
          'type': 'PlainText',
          'text': ''
        }
      },
      'shouldEndSession': 'true'
    },
    'sessionAttributes': {}
  }

def lambda_handler(event, context):
  if not is_valid_app(event):
    raise ValueError('Invalid Application ID')
  return get_response()
            

And that's pretty much all there is to it!

Issues/Future Improvements


Unfortunately, Alexa only consistently recognizes custom skill invocations when prefaced by a "open..." or "ask...", preventing me from simply saying "Alexa, what plane is that", which would be cooler. Scraping the route data also takes a long time, thereby delaying Alexa's response. I actually had to lengthen my AWS Lambda function's timeout to 10 seconds to ensure she wouldn't give up too soon. For Lambda noobs like myself, you can find the timeout setting in the Lambda menu shown below:


aws dashboard lambda settings

As for future improvements, it'd be cool to start storing metrics on which planes/flights pass by, and to be able to ask Alexa if I've seen a passing plane before. I'd also like the ability to ask follow up questions like how fast a passing plane is currently going.


✈️


All the code, including raspberry pi logic/server + Alexa skill, can be found on my Github (link below). Thanks for reading!