\begin{table}[t]
\centering
\begin{tabular}{c||ccc}
n-gram & match & Borderline & Facebook-AI \\ \hline \hline
demonstrators & 0.80 & 3 & 0 \\
the night & 0.80 & 3 & 0 \\
That & 0.80 & 3 & 0 \\
+ & 0.80 & 3 & 0 \\
in the night & 0.80 & 3 & 0 \\
discussed & 0.80 & 3 & 0 \\
theater & 0.80 & 3 & 0 \\
Naples & 0.80 & 3 & 0 \\
FC Chelsea & 0.80 & 3 & 0 \\
That is & 0.75 & 2 & 0 \\
unclear. & 0.75 & 2 & 0 \\
department & 0.75 & 2 & 0 \\
on Saturday in & 0.75 & 2 & 0 \\
- the & 0.75 & 2 & 0 \\
was on & 0.75 & 2 & 0 \\
lying & 0.75 & 2 & 0 \\
Interior Horst & 0.75 & 2 & 0 \\
the Interior Horst & 0.75 & 2 & 0 \\
the citizens & 0.75 & 2 & 0 \\
who have & 0.75 & 2 & 0 \\
Because of the & 0.75 & 2 & 0 \\
persons & 0.75 & 2 & 0 \\
the Interior Horst Seehofer & 0.75 & 2 & 0 \\
blues rock guitarist & 0.75 & 2 & 0 \\
fast and & 0.75 & 2 & 0 \\
Interior Horst Seehofer & 0.75 & 2 & 0 \\
fire department & 0.75 & 2 & 0 \\
quarantine. & 0.75 & 2 & 0 \\
Federal Minister & 0.75 & 2 & 0 \\
of the Interior Horst & 0.75 & 2 & 0 \\
the Interior & 0.75 & 2 & 0 \\
blues rock & 0.75 & 2 & 0 \\
of the Interior & 0.75 & 2 & 0 \\
crash of & 0.75 & 2 & 0 \\
travel & 0.75 & 2 & 0 \\
Federal Minister of & 0.75 & 2 & 0 \\
among & 0.75 & 2 & 0 \\
fight against the & 0.75 & 2 & 0 \\
Because of & 0.71 & 4 & 1 \\
crash & 0.71 & 4 & 1 \\
Because & 0.71 & 4 & 1 \\
the awarding of & 0.67 & 1 & 0 \\
a holding & 0.67 & 1 & 0 \\
to drive in & 0.67 & 1 & 0 \\
have been people who & 0.67 & 1 & 0 \\
club of & 0.67 & 1 & 0 \\
Wanderers. & 0.67 & 1 & 0 \\
and rolled over & 0.67 & 1 & 0 \\
Exactly & 0.67 & 1 & 0 \\
winner FC Bayern with & 0.67 & 1 & 0 \\
\end{tabular}
\caption{Caption}
\label{tab:table127}
\end{table}
50 n-grams where Facebook-AI>Borderline in match
n-gram
match
Borderline
Facebook-AI
50+1
0.17
0
4
Festival
0.20
0
3
the 50+1
0.20
0
3
landfall
0.25
0
2
lead through a
0.25
0
2
roller coaster
0.25
0
2
coaster
0.25
0
2
do
0.25
0
2
between two groups
0.25
0
2
the 50+1 rule.
0.25
0
2
roller
0.25
0
2
towards
0.25
0
2
lead through
0.25
0
2
Covid-19
0.25
0
2
being
0.25
0
2
up in
0.25
0
2
1:1
0.25
0
2
Frankfurt/Main
0.25
0
2
two groups of
0.25
0
2
Also
0.25
0
2
groups of
0.25
0
2
the lead through
0.25
0
2
in Austria.
0.25
0
2
a roller coaster
0.25
0
2
50+1 rule.
0.25
0
2
Thole/Wickler
0.25
0
2
for at
0.25
0
2
Covid-19.
0.25
0
2
the lead through a
0.25
0
2
passenger
0.25
0
2
made landfall
0.25
0
2
between two groups of
0.25
0
2
a roller
0.25
0
2
two groups
0.25
0
2
Donnersdorf/Steinsfeld
0.33
0
1
from the economy (construction,
0.33
0
1
held by
0.33
0
1
26, 2020, 8:00
0.33
0
1
the already
0.33
0
1
routing through the German
0.33
0
1
is held
0.33
0
1
been expelled
0.33
0
1
the already existing
0.33
0
1
curfew at 1
0.33
0
1
the short lead
0.33
0
1
the curfew at
0.33
0
1
Late-night
0.33
0
1
had now returned
0.33
0
1
were the third
0.33
0
1
a relatively strict
0.33
0
1
\begin{table}[t]
\centering
\begin{tabular}{c||ccc}
n-gram & match & Borderline & Facebook-AI \\ \hline \hline
50+1 & 0.17 & 0 & 4 \\
Festival & 0.20 & 0 & 3 \\
the 50+1 & 0.20 & 0 & 3 \\
landfall & 0.25 & 0 & 2 \\
lead through a & 0.25 & 0 & 2 \\
roller coaster & 0.25 & 0 & 2 \\
coaster & 0.25 & 0 & 2 \\
do & 0.25 & 0 & 2 \\
between two groups & 0.25 & 0 & 2 \\
the 50+1 rule. & 0.25 & 0 & 2 \\
roller & 0.25 & 0 & 2 \\
towards & 0.25 & 0 & 2 \\
lead through & 0.25 & 0 & 2 \\
Covid-19 & 0.25 & 0 & 2 \\
being & 0.25 & 0 & 2 \\
up in & 0.25 & 0 & 2 \\
1:1 & 0.25 & 0 & 2 \\
Frankfurt/Main & 0.25 & 0 & 2 \\
two groups of & 0.25 & 0 & 2 \\
Also & 0.25 & 0 & 2 \\
groups of & 0.25 & 0 & 2 \\
the lead through & 0.25 & 0 & 2 \\
in Austria. & 0.25 & 0 & 2 \\
a roller coaster & 0.25 & 0 & 2 \\
50+1 rule. & 0.25 & 0 & 2 \\
Thole/Wickler & 0.25 & 0 & 2 \\
for at & 0.25 & 0 & 2 \\
Covid-19. & 0.25 & 0 & 2 \\
the lead through a & 0.25 & 0 & 2 \\
passenger & 0.25 & 0 & 2 \\
made landfall & 0.25 & 0 & 2 \\
between two groups of & 0.25 & 0 & 2 \\
a roller & 0.25 & 0 & 2 \\
two groups & 0.25 & 0 & 2 \\
Donnersdorf/Steinsfeld & 0.33 & 0 & 1 \\
from the economy (construction, & 0.33 & 0 & 1 \\
held by & 0.33 & 0 & 1 \\
26, 2020, 8:00 & 0.33 & 0 & 1 \\
the already & 0.33 & 0 & 1 \\
routing through the German & 0.33 & 0 & 1 \\
is held & 0.33 & 0 & 1 \\
been expelled & 0.33 & 0 & 1 \\
the already existing & 0.33 & 0 & 1 \\
curfew at 1 & 0.33 & 0 & 1 \\
the short lead & 0.33 & 0 & 1 \\
the curfew at & 0.33 & 0 & 1 \\
Late-night & 0.33 & 0 & 1 \\
had now returned & 0.33 & 0 & 1 \\
were the third & 0.33 & 0 & 1 \\
a relatively strict & 0.33 & 0 & 1 \\
\end{tabular}
\caption{Caption}
\label{tab:table128}
\end{table}
Sentence Examples
10 sentences where Borderline>Facebook-AI at sentence-level BLEU
Output
sentbleu
Ref
Biker reacts lightning fast and prevents worse.
Borderline
Biker reacts lightning fast and prevents worse
83.66
Facebook-AI
Biker reacts with lightning speed and prevents worse
27.33
\begin{table}[t]
\centering
\begin{tabular}{c||cc}
& Output & sentbleu \\ \hline \hline
Ref & Biker reacts lightning fast and prevents worse. & \\
Borderline & Biker reacts lightning fast and prevents worse & 83.66 \\
Facebook-AI & Biker reacts with lightning speed and prevents worse & 27.33 \\
\end{tabular}
\caption{Caption}
\label{tab:table129}
\end{table}
Output
sentbleu
Ref
This decision would neither weaken the federal party nor its right wing.
Borderline
This decision would neither weaken the federal party nor the right wing.
78.97
Facebook-AI
Neither the federal party nor the right wing will be weakened by this decision.
25.03
\begin{table}[t]
\centering
\begin{tabular}{c||cc}
& Output & sentbleu \\ \hline \hline
Ref & This decision would neither weaken the federal party nor its right wing. & \\
Borderline & This decision would neither weaken the federal party nor the right wing. & 78.97 \\
Facebook-AI & Neither the federal party nor the right wing will be weakened by this decision. & 25.03 \\
\end{tabular}
\caption{Caption}
\label{tab:table130}
\end{table}
Output
sentbleu
Ref
Werth’s winning horse Quantaz belongs to Max-Theurer.
Borderline
Werth's winning horse Quantaz belongs to Max-Theurer.
83.66
Facebook-AI
Werth's winning horse Quantaz is owned by Max-Theurer.
\begin{table}[t]
\centering
\begin{tabular}{c||cc}
& Output & sentbleu \\ \hline \hline
Ref & New apartments are empty & \\
Borderline & New apartments are empty & 100.00 \\
Facebook-AI & New apartments stand empty & 50.00 \\
\end{tabular}
\caption{Caption}
\label{tab:table132}
\end{table}
Output
sentbleu
Ref
In the middle of the Blues Boom, Green founded Fleetwood Mac in the summer of 1967 together with his Bluesbreakers colleagues Mick Fleetwood and John McVie.
Borderline
In the middle of the blues boom, Green founded Fleetwood Mac in the summer of 1967 together with his Bluesbreakers colleagues Mick Fleetwood and John McVie.
85.84
Facebook-AI
In the midst of the blues boom, Green formed Fleetwood Mac in the summer of 1967 with fellow Bluesbreakers Mick Fleetwood and John McVie.
44.23
\begin{table}[t]
\centering
\begin{tabular}{c||cc}
& Output & sentbleu \\ \hline \hline
Ref & In the middle of the Blues Boom, Green founded Fleetwood Mac in the summer of 1967 together with his Bluesbreakers colleagues Mick Fleetwood and John McVie. & \\
Borderline & In the middle of the blues boom, Green founded Fleetwood Mac in the summer of 1967 together with his Bluesbreakers colleagues Mick Fleetwood and John McVie. & 85.84 \\
Facebook-AI & In the midst of the blues boom, Green formed Fleetwood Mac in the summer of 1967 with fellow Bluesbreakers Mick Fleetwood and John McVie. & 44.23 \\
\end{tabular}
\caption{Caption}
\label{tab:table133}
\end{table}
Output
sentbleu
Ref
The blues rock guitarist suffered from drug abuse and schizophrenia, as biographers report.
Borderline
The blues rock guitarist suffered from drug abuse and schizophrenia, as biographers reported.
91.80
Facebook-AI
The blues-rock guitarist suffered from drug abuse and schizophrenia, according to biographers.
52.50
\begin{table}[t]
\centering
\begin{tabular}{c||cc}
& Output & sentbleu \\ \hline \hline
Ref & The blues rock guitarist suffered from drug abuse and schizophrenia, as biographers report. & \\
Borderline & The blues rock guitarist suffered from drug abuse and schizophrenia, as biographers reported. & 91.80 \\
Facebook-AI & The blues-rock guitarist suffered from drug abuse and schizophrenia, according to biographers. & 52.50 \\
\end{tabular}
\caption{Caption}
\label{tab:table134}
\end{table}
Output
sentbleu
Ref
The numbers are still inconspicuous, said Terzenbach.
Borderline
The figures are still inconspicuous, said Terzenbach.
70.35
Facebook-AI
The figures are still unremarkable, said Terzenbach.
31.78
\begin{table}[t]
\centering
\begin{tabular}{c||cc}
& Output & sentbleu \\ \hline \hline
Ref & The numbers are still inconspicuous, said Terzenbach. & \\
Borderline & The figures are still inconspicuous, said Terzenbach. & 70.35 \\
Facebook-AI & The figures are still unremarkable, said Terzenbach. & 31.78 \\
\end{tabular}
\caption{Caption}
\label{tab:table135}
\end{table}
Output
sentbleu
Ref
In order to free the unfortunate driver, the fire department had to cut open the roof of the car.
Borderline
In order to save the victims, the fire department had to cut open the roof of the car.
70.30
Facebook-AI
In order to save the victim, the fire brigade had to cut the roof of the car free.
32.11
\begin{table}[t]
\centering
\begin{tabular}{c||cc}
& Output & sentbleu \\ \hline \hline
Ref & In order to free the unfortunate driver, the fire department had to cut open the roof of the car. & \\
Borderline & In order to save the victims, the fire department had to cut open the roof of the car. & 70.30 \\
Facebook-AI & In order to save the victim, the fire brigade had to cut the roof of the car free. & 32.11 \\
\end{tabular}
\caption{Caption}
\label{tab:table136}
\end{table}
Output
sentbleu
Ref
From July 31 on the 33 super sports cars will race through the Lausitz.
Borderline
From 31 July, the 33 super sports cars will race through Lusatia.
57.45
Facebook-AI
From 31 July, the 33 supercars will race through Lusatia.
19.91
\begin{table}[t]
\centering
\begin{tabular}{c||cc}
& Output & sentbleu \\ \hline \hline
Ref & From July 31 on the 33 super sports cars will race through the Lausitz. & \\
Borderline & From 31 July, the 33 super sports cars will race through Lusatia. & 57.45 \\
Facebook-AI & From 31 July, the 33 supercars will race through Lusatia. & 19.91 \\
\end{tabular}
\caption{Caption}
\label{tab:table137}
\end{table}
Output
sentbleu
Ref
Only 104 drivers started the final stage in the evening.
Borderline
Only 104 riders had entered the final stage in the evening.
56.63
Facebook-AI
Only 104 riders were left in the evening for the final stage.
19.73
\begin{table}[t]
\centering
\begin{tabular}{c||cc}
& Output & sentbleu \\ \hline \hline
Ref & Only 104 drivers started the final stage in the evening. & \\
Borderline & Only 104 riders had entered the final stage in the evening. & 56.63 \\
Facebook-AI & Only 104 riders were left in the evening for the final stage. & 19.73 \\
\end{tabular}
\caption{Caption}
\label{tab:table138}
\end{table}
10 sentences where Facebook-AI>Borderline at sentence-level BLEU
Output
sentbleu
Ref
The person had defected to South Korea three years ago and had now returned across the borders.
Borderline
The person fled to South Korea three years ago and has now returned across the border.
56.15
Facebook-AI
The person had defected to South Korea three years ago and had now returned across the border.
93.83
\begin{table}[t]
\centering
\begin{tabular}{c||cc}
& Output & sentbleu \\ \hline \hline
Ref & The person had defected to South Korea three years ago and had now returned across the borders. & \\
Borderline & The person fled to South Korea three years ago and has now returned across the border. & 56.15 \\
Facebook-AI & The person had defected to South Korea three years ago and had now returned across the border. & 93.83 \\
\end{tabular}
\caption{Caption}
\label{tab:table139}
\end{table}
Output
sentbleu
Ref
“We all liked to do that in our youth,” he said to the Ö1 radio broadcaster.
Borderline
"We all enjoyed doing this in our youth," he told the radio station Ö1.
12.20
Facebook-AI
"We all liked to do that in our youth," he told the Ö1 radio station.
50.46
\begin{table}[t]
\centering
\begin{tabular}{c||cc}
& Output & sentbleu \\ \hline \hline
Ref & “We all liked to do that in our youth,” he said to the Ö1 radio broadcaster. & \\
Borderline & "We all enjoyed doing this in our youth," he told the radio station Ö1. & 12.20 \\
Facebook-AI & "We all liked to do that in our youth," he told the Ö1 radio station. & 50.46 \\
\end{tabular}
\caption{Caption}
\label{tab:table140}
\end{table}
Output
sentbleu
Ref
Report: Seehofer pushes for longer telecommunications data retention
Borderline
Report: Seehofer urges longer data retention
27.22
Facebook-AI
Report: Seehofer pushes for longer data retention
66.33
\begin{table}[t]
\centering
\begin{tabular}{c||cc}
& Output & sentbleu \\ \hline \hline
Ref & Report: Seehofer pushes for longer telecommunications data retention & \\
Borderline & Report: Seehofer urges longer data retention & 27.22 \\
Facebook-AI & Report: Seehofer pushes for longer data retention & 66.33 \\
\end{tabular}
\caption{Caption}
\label{tab:table141}
\end{table}
Output
sentbleu
Ref
Baden-Württemberg: More than 250 illegal car races reported
Borderline
Baden-Württemberg: Over 250 illegal car races recorded
46.34
Facebook-AI
Baden-Württemberg: More than 250 illegal car races recorded
85.99
\begin{table}[t]
\centering
\begin{tabular}{c||cc}
& Output & sentbleu \\ \hline \hline
Ref & Baden-Württemberg: More than 250 illegal car races reported & \\
Borderline & Baden-Württemberg: Over 250 illegal car races recorded & 46.34 \\
Facebook-AI & Baden-Württemberg: More than 250 illegal car races recorded & 85.99 \\
\end{tabular}
\caption{Caption}
\label{tab:table142}
\end{table}
Output
sentbleu
Ref
There was no danger of a passing train.
Borderline
Danger by an incoming train did not exist.
0
Facebook-AI
There was no danger from an incoming train.
45.97
\begin{table}[t]
\centering
\begin{tabular}{c||cc}
& Output & sentbleu \\ \hline \hline
Ref & There was no danger of a passing train. & \\
Borderline & Danger by an incoming train did not exist. & 0 \\
Facebook-AI & There was no danger from an incoming train. & 45.97 \\
\end{tabular}
\caption{Caption}
\label{tab:table143}
\end{table}
Output
sentbleu
Ref
The record is held by Manchester City with 100 points in the season 2017/18.
Borderline
Manchester City holds the record with 100 points in the 2017 / 18 season.
37.05
Facebook-AI
The record is held by Manchester City with 100 points in the 2017/18 season.
84.84
\begin{table}[t]
\centering
\begin{tabular}{c||cc}
& Output & sentbleu \\ \hline \hline
Ref & The record is held by Manchester City with 100 points in the season 2017/18. & \\
Borderline & Manchester City holds the record with 100 points in the 2017 / 18 season. & 37.05 \\
Facebook-AI & The record is held by Manchester City with 100 points in the 2017/18 season. & 84.84 \\
\end{tabular}
\caption{Caption}
\label{tab:table144}
\end{table}
Output
sentbleu
Ref
A security service monitors the curfew.
Borderline
A security service monitors the ban on going out.
50.71
Facebook-AI
A security service monitors the curfew.
100.00
\begin{table}[t]
\centering
\begin{tabular}{c||cc}
& Output & sentbleu \\ \hline \hline
Ref & A security service monitors the curfew. & \\
Borderline & A security service monitors the ban on going out. & 50.71 \\
Facebook-AI & A security service monitors the curfew. & 100.00 \\
\end{tabular}
\caption{Caption}
\label{tab:table145}
\end{table}
Output
sentbleu
Ref
Climate activist Greta Thunberg was a guest via video connection in a popular US late-night show.
Borderline
Klimaatkivistin Greta Thunberg war per Videoschalte zu Gast in einer beliebten US-Latenacht-Show.
10.00
Facebook-AI
Climate activist Greta Thunberg was a guest on a popular US late-night show via video link.
59.78
\begin{table}[t]
\centering
\begin{tabular}{c||cc}
& Output & sentbleu \\ \hline \hline
Ref & Climate activist Greta Thunberg was a guest via video connection in a popular US late-night show. & \\
Borderline & Klimaatkivistin Greta Thunberg war per Videoschalte zu Gast in einer beliebten US-Latenacht-Show. & 10.00 \\
Facebook-AI & Climate activist Greta Thunberg was a guest on a popular US late-night show via video link. & 59.78 \\
\end{tabular}
\caption{Caption}
\label{tab:table146}
\end{table}
700 Deaths from Monsoon Rainfall in Southeast Asia
32.50
Facebook-AI
700 dead from monsoon rains in Southeast Asia
100.00
\begin{table}[t]
\centering
\begin{tabular}{c||cc}
& Output & sentbleu \\ \hline \hline
Ref & 700 dead from monsoon rains in Southeast Asia & \\
Borderline & 700 Deaths from Monsoon Rainfall in Southeast Asia & 32.50 \\
Facebook-AI & 700 dead from monsoon rains in Southeast Asia & 100.00 \\
\end{tabular}
\caption{Caption}
\label{tab:table148}
\end{table}
