Skip to content

Fixes and full results for bAbi RNN example #501

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
fchollet merged 3 commits into from
Aug 7, 2015
Merged

Fixes and full results for bAbi RNN example #501

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
63284a4
babi_rnn bugfix: QA19 requires vocab from the answer
Smerity Aug 6, 2015
dbc0c27
babi_rnn bugfix: Fixing missing Python 3 support
Smerity Aug 7, 2015
342f2bc
babi_rnn: Adding results for all tasks in bAbi tasks dataset
Smerity Aug 7, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
babi_rnn: Adding results for all tasks in bAbi tasks dataset
  • Loading branch information
@Smerity
Smerity committed Aug 7, 2015
commit 342f2bc271720a22c04e3986df4fc1530624a6cf
23 changes: 23 additions & 0 deletions examples/babi_rnn.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,29 @@
"Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks"
http://arxiv.org/abs/1502.05698

Task Number | FB LSTM Baseline | Keras QA
--- | --- | ---
QA1 - Single Supporting Fact | 50 | 52.1
QA2 - Two Supporting Facts | 20 | 37.0
QA3 - Three Supporting Facts | 20 | 20.5
QA4 - Two Arg. Relations | 61 | 62.9
QA5 - Three Arg. Relations | 70 | 61.9
QA6 - Yes/No Questions | 48 | 50.7
QA7 - Counting | 49 | 78.9
QA8 - Lists/Sets | 45 | 77.2
QA9 - Simple Negation | 64 | 64.0
QA10 - Indefinite Knowledge | 44 | 47.7
QA11 - Basic Coreference | 72 | 74.9
QA12 - Conjunction | 74 | 76.4
QA13 - Compound Coreference | 94 | 94.4
QA14 - Time Reasoning | 27 | 34.8
QA15 - Basic Deduction | 21 | 32.4
QA16 - Basic Induction | 23 | 50.6
QA17 - Positional Reasoning | 51 | 49.1
QA18 - Size Reasoning | 52 | 90.8
QA19 - Path Finding | 8 | 9.0
QA20 - Agent's Motivations | 91 | 90.7

For the resources related to the bAbI project, refer to:
https://research.facebook.com/researchers/1543934539189348

Expand Down