precision | recall | f1-score | support | |
---|---|---|---|---|
CALLSIGN | 0.72 | 1.00 | 0.84 | 34 |
NOCALLSIGN | 0.00 | 0.00 | 0.00 | 13 |
Accuracy | 0.72 | 47 | ||
Macro avg | 0.36 | 0.5 | 0.42 | 47 |
Weighted avg | 0.52 | 0.72 | 0.61 | 47 |
precision | recall | f1-score | support | |
---|---|---|---|---|
CALLSIGN | 0.75 | 0.88 | 0.81 | 34 |
NOCALLSIGN | 0.43 | 0.23 | 0.30 | 13 |
Accuracy | 0.70 | 47 | ||
Macro avg | 0.59 | 0.56 | 0.56 | 47 |
Weighted avg | .66 | .70 | .67 | 47 |
precision | recall | f1-score | support | |
---|---|---|---|---|
CLIMB | 1 | 0.62 | 0.77 | 8 |
CONTACT | 1 | 1 | 1 | 4 |
DESCEND | 1 | 1 | 1 | 3 |
DIRECTTO | 1 | 0.5 | 0.67 | 4 |
PROCEEDDIRECT | 0.8 | 1 | 0.89 | 4 |
PROCEED | 1 | 1 | 1 | 1 |
RHEIN | 1 | 0.86 | 0.92 | 7 |
RHEINRADAR | 0.94 | 1 | 0.97 | 15 |
SETCOURSE | 0.43 | 1 | 0.6 | 3 |
SET | 0 | 0 | 0 | 0 |
TURN | 0.8 | 0.67 | 0.73 | 6 |
TURNHEADING | 0 | 0 | 0 | 2 |
TURNLEFT | 0.71 | 1 | 0.83 | 5 |
TURNRIGHT | 0.8 | 1 | 0.89 | 4 |
micro avg | 0.85 | 0.85 | 0.85 | 66 |
macro avg | 0.75 | 0.76 | 0.73 | 66 |
weighted avg | 0.87 | 0.85 | 0.84 | 66 |
DESCEND FLIGHT TWO ZERO EXPEDITE DESCENT AS CROSSING TRAFFIC | [“DESCEND”] |
DESCEND FLIGHT LEVEL THREE EIGHT ZERO | [“DESCEND”] |
DESCEND FLIGHT LEVEL THREE FO= SEVEN | [“DESCEND”] |
DESCEND FLIGHT LEVEL THREE HUNDRED RATE ONE THOUSAND FIVE HUNDRED OR MORE | [“DESCEND”] |
DESCEND FLIGHT LEVEL THREE ONE TWO FIVE | [“DESCEND”] |
DESCEND FLIGHT LEVEL THREE ONE TWO | [“DESCEND”] |
DESCEND FLIGHT LEVEL THREE ONE ZERO | [“DESCEND”] |
DESCEND FLIGHT LEVEL THREE TWO ZERO RATE FIFTEEN HUNDRED OR MORE | [“DESCEND”] |
DESCEND FLIGHT LEVEL THREE TWO ZERO | [“DESCEND”] |
DESCEND FLIGHT LEVEL TO YOU ZERO SET COURSE TO FOXTROT | [“DESCEND”] |
CLIMB FLIGHT LEVEL THREE HUNDRED SET COURSE | [“CLIMB”] |
CLIMB FLIGHT LEVEL TWO NINE ALSO SET COURSE TRASADINGEN | [“CLIMB”] |
CONTACT ~M YOU ONE ONE NATTENHEIM ONE ONE OF TWO | [“CONTACT”] |
CONTACT ONE ONE ~M ONE AND | [“CONTACT”] |
CONTACT RHEIN ONE THREE TWO DECIMAL FOUR ~S | [“CONTACT”] |
DESCEND FLIGHT SET EIGHT ONE SIX LEFT | [“DESCEND”] |
DIRECT TO FOXTROT ROMEO INDIA | [“DIRECTTO”] |
RHEIN ~M MINUTE AND ~M ONE ONE ONE | [“RHEIN”] |
RHEIN AH AND YOU ONE AH GUTEN | [“RHEIN”] |
RHEIN AIR ONE THREE TWO DECIMAL FOUR | [“RHEIN”] |
RHEIN ONE AND CALL | [“RHEIN”] |
Algorithm | Callsign Detection | No Callsign Detection | Chart Representation |
---|---|---|---|
Logistic Regression | 63% | 86% | |
LSVM | 49% | 87% | |
Naïve Bayes | 43% | 91% |
Work | Technique | Command Error Recognition Rate |
---|---|---|
Helmke et al. [ |
SLM+Rescoring (N-best=5) | 16.5 |
Helmke et al. [ |
ASR-SSA-C Concept | 35.9 |
This work | Syntactic Separation for Callsign and Rest of Callsign and using ML classification algorithms | 16% (callsign best) 9% (rest of callsign best) |
Callsign | No Callsign |
---|---|
AERO LLOYD FIVE FIVE FIVE | [“CALLSIGN”] |
AERO LLOYD FIVE FIVE NINE | [“CALLSIGN”] |
AERO LLOYD FIVE FIVE ONE | [“CALLSIGN”] |
AERO LLOYD FIVE FIVE SEVEN | [“CALLSIGN”] |
AERO LLOYD FIVE FIVE TWO | [“CALLSIGN”] |
AERO LLOYD ONE ONE FOUR | [“CALLSIGN”] |
AERO LLOYD TRY A TWO | [“CALLSIGN”] |
AERO LLOYD TWO TWO SIX | [“CALLSIGN”] |
CLIMB FLIGHT LEVEL THREE FIVE SINGA SET COURSE TO COURSE TO | [“SETCOURSE”] |
CLIMB FLIGHT LEVEL THREE HUNDRED SET COURSE | [“SETCOURSE”] |
CLIMB FLIGHT LEVEL TWO NINE ALSO SET COURSE TRASADINGEN | [“SETCOURSE”] |
RHEIN IDENT YOU READ | [“RHEIN”] |
RHEIN IDENTIFIED CLIMB FLIGHT LEVEL THREE YOU ARE YOU STILL | [“RHEINRADAR”] |
RHEIN IDENTIFIED CLIMB FLIGHT LEVEL TWO NINE ALSO SET COSTA THANK YOU | [“RHEINRADAR”] |
RHEIN IDENTIFIED CLIMB FLIGHT LEVEL TWO NINE ALSO | [“RHEINRADAR”] |
RHEIN IDENTIFIED CLIMB FLIGHT ONE IDENTIFIED | [“RHEINRADAR”] |
RHEIN IDENTIFIED CLIMB FLIGHT TO TWO SEVEN ZERO | [“RHEINRADAR”] |
HEADING FLIGHT HUNDRED MILANO ONE ONE THREE FOUR DECIMAL FIVE TWO | [“NOCALLSIGN”] |
HEADING OLYMPIC ONE ONE | [“NOCALLSIGN”] |
HEADING ONE ONE ONE ONE | [“NOCALLSIGN”] |
HEADING TWO TWO SIX FOUR | [“NOCALLSIGN”] |
HEADING YOU CHARLIE PAPA PAPA | [“NOCALLSIGN”] |
HIGHER HIGHER | [“NOCALLSIGN”] |
HIGHER ONE ONE ONE ONE | [“NOCALLSIGN”] |
HOTEL FIVE FIVE SINGA TWO | [“CALLSIGN”] |
HOTEL HOTEL | [“CALLSIGN”] |
Algorithm | Multi Classifier Accuracy | Chart Representation |
---|---|---|
Logistic Regression | 84% | |
LSVM | 83% | |
Naïve Bayes | 58% |
precision | recall | f1-score | support | |
---|---|---|---|---|
CLIMB | 0.8 | 1 | 0.89 | 8 |
CONTACT | 0.57 | 1 | 0.73 | 4 |
DESCEND | 1 | 0.33 | 0.5 | 3 |
DIRECTTO | 0 | 0 | 0 | 4 |
PROCEEDDIRECT | 0 | 0 | 0 | 4 |
PROCEED | 0 | 0 | 0 | 1 |
RHEIN | 0 | 0 | 0 | 7 |
RHEINRADAR | 0.67 | 0.93 | 0.78 | 15 |
SETCOURSE | 0.25 | 1 | 0.4 | 3 |
SET | 0 | 0 | 0 | 0 |
TURN | 0 | 0 | 0 | 6 |
TURNHEADING | 0 | 0 | 0 | 2 |
TURNLEFT | 0.4 | 0.8 | 0.53 | 5 |
TURNRIGHT | 0.8 | 1 | 0.89 | 4 |
Micro avg | 0.58 | 0.58 | 0.58 | 66 |
Macro avg | 0.32 | 0.43 | 0.34 | 66 |
Weighted avg | 0.42 | 0.58 | 0.46 | 66 |
precision | recall | f1-score | support | |
---|---|---|---|---|
CLIMB | 1 | 0.62 | 0.77 | 8 |
CONTACT | 1 | 1 | 1 | 4 |
DESCEND | 1 | 1 | 1 | 3 |
DIRECTTO | 1 | 0.25 | 0.4 | 4 |
PROCEEDDIRECT | 0.8 | 1 | 0.89 | 4 |
PROCEED | 1 | 1 | 1 | 1 |
RHEIN | 1 | 0.86 | 0.92 | 7 |
RHEINRADAR | 0.94 | 1 | 0.97 | 15 |
SETCOURSE | 0.38 | 1 | 0.55 | 3 |
SET | 0 | 0 | 0 | 0 |
TURN | 0.8 | 0.67 | 0.73 | 6 |
TURNHEADING | 0 | 0 | 0 | 2 |
TURNLEFT | 0.83 | 1 | 0.91 | 5 |
TURNRIGHT | 0.67 | 1 | 0.8 | 4 |
micro avg | 0.83 | 0.83 | 0.83 | 66 |
macro avg | 0.74 | 0.74 | 0.71 | 66 |
weighted avg | 0.86 | 0.83 | 0.82 | 66 |
precision | recall | f1-score | support | |
---|---|---|---|---|
CALLSIGN | 0.75 | 0.97 | 0.85 | 34 |
NOCALLSIGN | 0.67 | 0.15 | 0.25 | 13 |
Accuracy | 0.74 | 37 | ||
Macro avg | 0.71 | 0.56 | 0.55 | 47 |
Weighted avg | 0.73 | 0.74 | 0.68 | 47 |
FOUR FOUR SINGA TWO | [“NOCALLSIGN”] |
FOUR FOUR | [“NOCALLSIGN”] |
FOX JET JET | [“NOCALLSIGN”] |
FOX READ FIVE | [“NOCALLSIGN”] |
FOX SWISSAIR SWISSAIR | [“NOCALLSIGN”] |
FOXTROT CHECK RADIO CHECK JET | [“NOCALLSIGN”] |