Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
210 commits
Select commit Hold shift + click to select a range
b8d94ca
option to test on entire dataset
ArshdeepSekhon Oct 22, 2020
c863095
eval on entire dataset, checks
ArshdeepSekhon Oct 22, 2020
13bc18a
fix failed checks
ArshdeepSekhon Oct 22, 2020
8c726d1
Update 0_End_to_End.ipynb
tk-dev11 Oct 23, 2020
beac247
Merge pull request #314 from tahmid-kazi/patch-1
qiyanjun Oct 24, 2020
74bc5c0
add sphinx autodoc generated
qiyanjun Oct 24, 2020
0c95966
Update conf.py
qiyanjun Oct 24, 2020
dfe26b6
Update conf.py
qiyanjun Oct 24, 2020
e91a652
Two versions of API references now
qiyanjun Oct 24, 2020
d326796
Merge pull request #315 from QData/new_doc
qiyanjun Oct 24, 2020
688d997
Update conf.py
qiyanjun Oct 25, 2020
1e36c63
add in "sphinx.ext.autosummary"
qiyanjun Oct 25, 2020
821c350
move the mergign of api and source two version docs to branch new_doc
qiyanjun Oct 25, 2020
fc7d529
move documents from semi-automated rst files to correct code positions
qiyanjun Oct 25, 2020
2ea690b
a major shift to rst files generated by sphinx-apidoc
qiyanjun Oct 26, 2020
00c80ee
Update transformation.rst
qiyanjun Oct 26, 2020
95603fb
Merge branch 'master' into new_doc
qiyanjun Oct 26, 2020
23d0de6
clean up the docstring of each module and module content section
qiyanjun Oct 26, 2020
c41a13e
Update textattack.rst
qiyanjun Oct 26, 2020
c78aa66
fix all flake8 found whitespacing trailing issues
qiyanjun Oct 26, 2020
a4bd404
remove submodule lines from autodoc rsts
qiyanjun Oct 26, 2020
69a415f
correct two formatting with black
qiyanjun Oct 26, 2020
b998e0f
correct docstring style erors
qiyanjun Oct 26, 2020
887dca1
Update minimize_bleu.py
qiyanjun Oct 26, 2020
36dfce6
Merge pull request #316 from QData/new_doc
qiyanjun Oct 26, 2020
77ff8f3
Installation link fix
dheerajrav Oct 26, 2020
4c69e86
Merge pull request #317 from dheerajrav/patch-1
qiyanjun Oct 27, 2020
96a0360
remove the old API docs
qiyanjun Oct 29, 2020
6d83b2c
isort format of attack_camembert
qiyanjun Oct 29, 2020
230e39b
solve flake8 warning
qiyanjun Oct 29, 2020
f794639
Temporary Fix of Flair Testing Error
Hanyu-Liu-123 Oct 29, 2020
2509a14
Fix Format Error
Hanyu-Liu-123 Oct 29, 2020
58f522b
Update test_misc.py
Hanyu-Liu-123 Oct 29, 2020
af029bf
Specify torch version
Hanyu-Liu-123 Oct 29, 2020
41e7472
Merge pull request #322 from QData/new_docs
qiyanjun Oct 29, 2020
7dcf27d
add lazy loading
jinyongyoo Oct 28, 2020
8d64e89
formatting
jinyongyoo Oct 28, 2020
4478ad9
remove unncessary future statements
jinyongyoo Oct 29, 2020
3458a2b
rebase and formatting
jinyongyoo Oct 30, 2020
e6786b6
revert changes and fix circular imports
jinyongyoo Oct 30, 2020
7a89aba
make some dependencies optional
jinyongyoo Oct 30, 2020
dcce7b3
undo edits
jinyongyoo Oct 30, 2020
46ef3bc
fix github flow and import issues
jinyongyoo Oct 31, 2020
3c65e9d
remove 3.9 from testing
jinyongyoo Oct 31, 2020
4f16bb0
fix tqdm version conflict with datasets
jinyongyoo Oct 31, 2020
e2dd750
add install of stanza resource when package is present
jinyongyoo Oct 31, 2020
2149646
add embedding layer retrieval to models
jinyongyoo Oct 5, 2020
3ffd776
add get_grad method
jinyongyoo Oct 5, 2020
6273b19
make WordSwapGradientBased work with get_grad
jinyongyoo Oct 5, 2020
bdbeae8
add gradient-based white-box search
jinyongyoo Oct 5, 2020
79f3f4f
add new test for gradient-based greedy-word-wir
jinyongyoo Oct 5, 2020
4893d47
add consistent is_black_box attributes
jinyongyoo Oct 6, 2020
dcf408b
fix wrong test output
jinyongyoo Oct 17, 2020
84c1681
update gradient test to be device invariant
jinyongyoo Oct 17, 2020
866301b
fix bugs and revise word to token matching scheme
jinyongyoo Nov 1, 2020
8dad54a
fix test outputs and bugs
jinyongyoo Nov 1, 2020
a0e9479
WordSwapChangeName transformation now accounts
dheerajrav Nov 6, 2020
07abaf5
Merge pull request #319 from dheerajrav/word_swap_change_fix
jinyongyoo Nov 6, 2020
d9ae5f6
added table with attack recipes
tk-dev11 Nov 6, 2020
5325ef6
allow * and _ and @ in a word when tokenizing
tsinggggg Nov 6, 2020
e6f07eb
add comments
tsinggggg Nov 6, 2020
740c333
Merge pull request #327 from tsinggggg/allow-words-connected-by-addit…
jinyongyoo Nov 6, 2020
c04f8f4
Merge pull request #292 from QData/gradient-support
jxmorris12 Nov 6, 2020
6ccfaea
turn unknown goal function error into a logger warning
Nov 7, 2020
16c3b0d
update package versions
jinyongyoo Nov 8, 2020
acbc6c0
update compatibility check with current modules
jinyongyoo Nov 8, 2020
03e85b3
Merge pull request #330 from QData/update-package-version
jinyongyoo Nov 8, 2020
fb6088f
Merge pull request #329 from a1noack/new_goal_fcns
jxmorris12 Nov 9, 2020
16986b4
Merge pull request #326 from tahmid-kazi/patch-2
qiyanjun Nov 10, 2020
f950128
add a single doc page expaining our benchmarking results on search me…
qiyanjun Nov 10, 2020
de0b5b8
add in the docstring for textattack_cli main func
qiyanjun Nov 10, 2020
0f0649e
Merge pull request #331 from QData/new_docs
qiyanjun Nov 10, 2020
2328b7f
Merge branch 'master' into lazy-loading
jinyongyoo Nov 11, 2020
b148bd2
fix bugs
jinyongyoo Nov 11, 2020
ad68251
Merge pull request #323 from QData/lazy-loading
jinyongyoo Nov 11, 2020
f44947d
Update word_swap_embedding.py
qiyanjun Nov 11, 2020
2df8833
fix training code bug
jinyongyoo Nov 11, 2020
a9ab5d4
Merge pull request #332 from QData/fix-training-bug
jxmorris12 Nov 11, 2020
7caab13
support custom embeddings through gensim; incorporate WordEmbedding c…
tsinggggg Nov 12, 2020
170e45b
fix inflection generation and enlarge pos to upos mapping
Nov 12, 2020
e04c377
Fix for tut0
Oct 15, 2020
f0e27d1
Change model to Huggingface
Oct 25, 2020
3e9cbcb
Remove extras
Nov 12, 2020
be6cd88
remove duplicate keys from and fix formatting pos mapping dict
Nov 12, 2020
24669a9
fix formatting
a1noack Nov 12, 2020
b21bdc9
Remove model wrapping from pytorch wrapper module
Nov 12, 2020
305eb48
isort imports
a1noack Nov 12, 2020
6838209
run isort v5
a1noack Nov 12, 2020
4dc804a
Update requirements.txt
jinyongyoo Nov 13, 2020
e33bfc2
Merge pull request #334 from a1noack/fix_morpheus
jinyongyoo Nov 13, 2020
b0892ce
Remove model wrapping module
Nov 13, 2020
a03cbb1
Merge pull request #335 from QData/update-requirements.txt
jxmorris12 Nov 13, 2020
5f04781
Merge pull request #303 from sanchit97/tut0-fix
qiyanjun Nov 13, 2020
e583188
separate different types of word embeddings into differetn classes
jinyongyoo Nov 13, 2020
d810489
Fix ReadMe Minor Mistakes
Hanyu-Liu-123 Nov 13, 2020
155dd49
Merge pull request #336 from QData/Hanyu-Liu-123-patch-1
qiyanjun Nov 13, 2020
6bf376f
v0.2.12
jxmorris12 Nov 13, 2020
0401591
Update word_swap_embedding.py
qiyanjun Nov 11, 2020
a0193ba
Merge branch 'new_docs' of https://github.com/QData/TextAttack into n…
qiyanjun Nov 13, 2020
9bacdfc
add in overview diagram and models.md into doc
qiyanjun Nov 13, 2020
5c1579e
revise table of recipe-args in Readme.md
qiyanjun Nov 13, 2020
152a8d8
Update README.md
qiyanjun Nov 13, 2020
c0a60b3
share word embedding across components
jinyongyoo Nov 13, 2020
bbd5a7e
Update README.md
qiyanjun Nov 13, 2020
8bf85fd
Update README.md
qiyanjun Nov 13, 2020
aeacb55
combine with Sanchit's notebbook udpate
qiyanjun Nov 13, 2020
56d5bc5
add Glossary index
qiyanjun Nov 13, 2020
cff1d2d
Update README.md
qiyanjun Nov 14, 2020
18ecc5d
Update README.md
qiyanjun Nov 14, 2020
c8047d6
Update README.md
qiyanjun Nov 14, 2020
783213f
Update README.md
qiyanjun Nov 14, 2020
ea191ca
Update README.md
qiyanjun Nov 14, 2020
3abaae4
Update README.md
qiyanjun Nov 14, 2020
84ab6bc
Update README.md
qiyanjun Nov 14, 2020
c2924a4
disable parallel tests
jinyongyoo Nov 14, 2020
7a27040
Update README.md
qiyanjun Nov 14, 2020
e7484d7
Update README.md
qiyanjun Nov 14, 2020
9ee0ba1
Update README.md
qiyanjun Nov 14, 2020
0ee2f68
Update README.md
qiyanjun Nov 14, 2020
bb4539a
get rid of glossary index... not quite useful
qiyanjun Nov 14, 2020
48d89d6
Merge pull request #338 from QData/new_docs
qiyanjun Nov 14, 2020
0fad7f1
fix readthedocs module load
jinyongyoo Nov 14, 2020
a7ffdd2
local check pull 339, every doc builds well. besides add the attacks4…
qiyanjun Nov 14, 2020
5214d3b
Merge pull request #339 from QData/fix-google-dep
qiyanjun Nov 14, 2020
f1c3221
Update attacks4Components.md
qiyanjun Nov 14, 2020
b14f753
Merge pull request #340 from QData/fix-google-dep
qiyanjun Nov 14, 2020
7f2621b
move model-zoo.md into the index intro section
qiyanjun Nov 14, 2020
a1781a0
resolve the double indexing issues in the attack_recipes.rst
qiyanjun Nov 14, 2020
1295e8f
Correct URL issues in the Table of Four Components for 16 Attack Recipes
qiyanjun Nov 16, 2020
0adaa2b
add FAQ.md / fix docstring error / add links to two other textAttack …
qiyanjun Nov 17, 2020
b9e0c1d
Merge pull request #343 from QData/new_docs
qiyanjun Nov 17, 2020
04e98ba
update the textAttack paper abstract to the most-recent-version
qiyanjun Nov 17, 2020
c214210
Merge pull request #344 from QData/new_docs
qiyanjun Nov 17, 2020
1e65a93
correct the URL RST format used in references.md
qiyanjun Nov 17, 2020
c77f266
correct the URL RST format used in references.md
qiyanjun Nov 17, 2020
2215c56
Merge branch 'new_docs' of https://github.com/QData/TextAttack into n…
qiyanjun Nov 17, 2020
f375f87
Merge pull request #345 from QData/new_docs
qiyanjun Nov 17, 2020
061bbd9
add in the categorized-attacks.png and try to complete the documentation
qiyanjun Nov 17, 2020
d320b8d
Image Link errors (minor)
qiyanjun Nov 17, 2020
9ead853
Merge pull request #346 from QData/docs
qiyanjun Nov 17, 2020
5cd97aa
Update textattack_ecosystem.png
qiyanjun Nov 17, 2020
61b4d56
v0.2.13
jxmorris12 Nov 18, 2020
3bdd0a8
roll back transformers versions
jxmorris12 Nov 18, 2020
71185a0
v0.2.14
jxmorris12 Nov 18, 2020
e00253e
Create 3_Augmentations.ipynb
Hanyu-Liu-123 Nov 18, 2020
168aeda
add in "model card" information in the modles.md documentation.
qiyanjun Nov 18, 2020
02c653d
Merge branch 'master' of https://github.com/QData/TextAttack
qiyanjun Nov 18, 2020
993d49e
connect Readme.md in model folder to https://textattack.readthedocs.i…
qiyanjun Nov 18, 2020
ef3c2e2
update pos tagging in AttackedText
jinyongyoo Nov 19, 2020
ede35d4
fix query count when no candidate result score values are better than…
Nov 19, 2020
a53aec9
Update index.rst
Hanyu-Liu-123 Nov 20, 2020
1096541
Merge pull request #349 from QData/update-pos-tagging
jinyongyoo Nov 20, 2020
2719e20
Merge pull request #347 from QData/notebook
qiyanjun Nov 20, 2020
fe5aa5c
fix query count for GoalFunctionResult returned by SearchMethod._perf…
Nov 21, 2020
d37c5ab
remove duplicate line
Opdoop Nov 21, 2020
ec26dbf
make sure contents in the readme.md and readthedoc consistent
qiyanjun Nov 21, 2020
695d00a
Merge pull request #353 from Opdoop/patch-2
qiyanjun Nov 21, 2020
9d7c9ba
Update README.md
qiyanjun Nov 21, 2020
2fa6544
add structure in the FAQ
qiyanjun Nov 21, 2020
46ab64f
emphasize the point on constraint hold constant for Benchmarking Atta…
qiyanjun Nov 21, 2020
545687d
fix search behavior and update test
jinyongyoo Nov 22, 2020
278bb33
Merge pull request #350 from a1noack/fix_query_count
jinyongyoo Nov 22, 2020
eb0b837
Add chinese readme document
Opdoop Nov 19, 2020
9b1898f
Add the chinese version of readme
Opdoop Nov 20, 2020
8195194
Modified minor format of chinese version readme
Opdoop Nov 20, 2020
a7759a0
Modified table format of chinese version readme
Opdoop Nov 20, 2020
0f8abe7
Modified table format of chinese version readme
Opdoop Nov 20, 2020
45a1375
Modified table format of chinese version readme
Opdoop Nov 20, 2020
b71461b
Modified table format of chinese version readme
Opdoop Nov 20, 2020
b3b4896
Delete html table inline style
Opdoop Nov 20, 2020
e10efaf
Add DIV block to table title
Opdoop Nov 20, 2020
9098f89
test markdown colspan and breakline
Opdoop Nov 20, 2020
d0a6abf
test markdown colspan and breakline
Opdoop Nov 20, 2020
5a345e6
test markdown colspan and breakline
Opdoop Nov 20, 2020
c9451e3
test markdown colspan and breakline
Opdoop Nov 20, 2020
5b7ddc2
format html table width by adding slash
Opdoop Nov 20, 2020
d8b2c9d
Delete markdowntable_test.md
Opdoop Nov 20, 2020
987a0a2
made minor grammar modifications to the chinese readme
MintForever Nov 22, 2020
28601eb
Add chinese readme document
Opdoop Nov 19, 2020
8f403d4
Add the chinese version of readme
Opdoop Nov 20, 2020
d89c81c
Modified minor format of chinese version readme
Opdoop Nov 20, 2020
3f3b390
Modified table format of chinese version readme
Opdoop Nov 20, 2020
8580d0e
Modified table format of chinese version readme
Opdoop Nov 20, 2020
f503eb1
Modified table format of chinese version readme
Opdoop Nov 20, 2020
5dcd2a0
Modified table format of chinese version readme
Opdoop Nov 20, 2020
3997933
Delete html table inline style
Opdoop Nov 20, 2020
167e0bc
Add DIV block to table title
Opdoop Nov 20, 2020
0a22412
test markdown colspan and breakline
Opdoop Nov 20, 2020
ba44ea9
test markdown colspan and breakline
Opdoop Nov 20, 2020
2266394
test markdown colspan and breakline
Opdoop Nov 20, 2020
30a2bc9
test markdown colspan and breakline
Opdoop Nov 20, 2020
f819d8a
format html table width by adding slash
Opdoop Nov 20, 2020
c5139c2
Delete markdowntable_test.md
Opdoop Nov 20, 2020
318d7ab
Remote html style and class
Opdoop Nov 23, 2020
8c8b68f
Remote html style and class
Opdoop Nov 23, 2020
f617feb
Remote class
Opdoop Nov 23, 2020
8dc04ac
Merge remote-tracking branch 'TextAttack/review_readme_zh' into readm…
Opdoop Nov 24, 2020
8cfb885
Change the minor formation of html table
Opdoop Nov 24, 2020
fdfaf6c
Update readme_zh design section
Opdoop Nov 25, 2020
6f12980
add in README_ZH.md into the main README
qiyanjun Nov 25, 2020
6a383bf
Merge pull request #352 from Opdoop/readme_zh
qiyanjun Nov 25, 2020
e195181
rename classes
jinyongyoo Nov 27, 2020
43e7577
Merge pull request #333 from tsinggggg/custom-word-embedding
jinyongyoo Nov 27, 2020
4195868
option to test on entire dataset
ArshdeepSekhon Oct 22, 2020
754e06a
eval on entire dataset, checks
ArshdeepSekhon Oct 22, 2020
54e01ff
fix failed checks
ArshdeepSekhon Oct 22, 2020
b29a09a
evaluate on entire dataset
ArshdeepSekhon Nov 27, 2020
50baac6
evaluate on entire dataset
ArshdeepSekhon Nov 27, 2020
c55dd28
remove unused test-on-entire-dataset option
ArshdeepSekhon Nov 27, 2020
7635c7b
modify attack accordingly for entire dataset
ArshdeepSekhon Nov 27, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
correct the URL RST format used in references.md
  • Loading branch information
qiyanjun committed Nov 17, 2020
commit 1e65a93c8c33675e39845c46c3daa725686cbf66
17 changes: 9 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ or via python module (`python -m textattack ...`).
> dataset samples, and the configuration file `config.yaml`. To change the cache path, set the
> environment variable `TA_CACHE_DIR`. (for example: `TA_CACHE_DIR=/tmp/ textattack attack ...`).

## Usage `textattack --help`
## Usage: `textattack --help`

TextAttack's main features can all be accessed via the `textattack` command. Two very
common commands are `textattack attack <args>`, and `textattack augment <args>`. You can see more
Expand All @@ -71,7 +71,7 @@ textattack attack --help

The [`examples/`](examples/) folder includes scripts showing common TextAttack usage for training models, running attacks, and augmenting a CSV file. The [documentation website](https://textattack.readthedocs.io/en/latest) contains walkthroughs explaining basic usage of TextAttack, including building a custom transformation and a custom constraint..

### Running Attacks `textattack attack --help`
### Running Attacks: `textattack attack --help`

The easiest way to try out an attack is via the command-line interface, `textattack attack`.

Expand Down Expand Up @@ -99,7 +99,7 @@ textattack attack --model lstm-mr --num-examples 20 \

> **Tip:** Instead of specifying a dataset and number of examples, you can pass `--interactive` to attack samples inputted by the user.

### Attacks and Papers Implemented ("Attack Recipes") `textattack attack --recipe [recipe_name]`
### Attacks and Papers Implemented ("Attack Recipes"): `textattack attack --recipe [recipe_name]`

We include attack recipes which implement attacks from the literature. You can list attack recipes using `textattack list attack-recipes`.

Expand Down Expand Up @@ -291,7 +291,7 @@ textattack attack --model bert-base-uncased-sst2 --recipe textfooler --num-examp
textattack attack --model t5-en-de --recipe seq2sick --num-examples 100
```

### Augmenting Text `textattack augment`
### Augmenting Text: `textattack augment`

Many of the components of TextAttack are useful for data augmentation. The `textattack.Augmenter` class
uses a transformation and a list of constraints to augment data. We also offer five built-in recipes
Expand Down Expand Up @@ -365,7 +365,7 @@ You can also create your own augmenter from scratch by importing transformations
['What I cannot creae, I do not understand.', 'What I cannot creat, I do not understand.', 'What I cannot create, I do not nderstand.', 'What I cannot create, I do nt understand.', 'Wht I cannot create, I do not understand.']
```

### Training Models `textattack train`
### Training Models: `textattack train`

Our model training code is available via `textattack train` to help you train LSTMs,
CNNs, and `transformers` models using TextAttack out-of-the-box. Datasets are
Expand All @@ -388,12 +388,13 @@ This uses the `EasyDataAugmenter` recipe to augment the `rotten_tomatoes` datase
textattack train --model bert-base-uncased --dataset glue^cola --batch-size 32 --epochs 5
```

### `textattack peek-dataset`

### To check datasets: `textattack peek-dataset`

To take a closer look at a dataset, use `textattack peek-dataset`. TextAttack will print some cursory statistics about the inputs and outputs from the dataset. For example, `textattack peek-dataset --dataset-from-huggingface snli` will show information about the SNLI dataset from the NLP package.


### To list components `textattack list`
### To list components: `textattack list`

There are lots of pieces in TextAttack, and it can be difficult to keep track of all of them. You can use `textattack list` to list components, for example, pretrained models (`textattack list models`) or available search methods (`textattack list search-methods`).

Expand Down Expand Up @@ -474,7 +475,7 @@ You can then run attacks on samples from this dataset by adding the argument `--



### Attacks
### Attacks and how to design a new attack

The `attack_one` method in an `Attack` takes as input an `AttackedText`, and outputs either a `SuccessfulAttackResult` if it succeeds or a `FailedAttackResult` if it fails.

Expand Down
8 changes: 4 additions & 4 deletions docs/1start/references.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ How to Cite TextAttack

- Paper [EMNLP Demo](https://arxiv.org/abs/2005.05909)

- Abstract: While there has been substantial research using adversarial attacks to analyze NLP models, each attack is implemented in its own code repository. It remains challenging to develop NLP attacks and utilize them to improve model performance. This paper introduces TextAttack, a Python framework for adversarial attacks, data augmentation, and adversarial training in NLP. TextAttack builds attacks from four components: a goal function, a set of constraints, a transformation, and a search method. TextAttack's modular design enables researchers to easily construct attacks from combinations of novel and existing components. TextAttack provides implementations of 16 adversarial attacks from the literature and supports a variety of models and datasets, including BERT and other transformers, and all GLUE tasks. TextAttack also includes data augmentation and adversarial training modules for using components of adversarial attacks to improve model accuracy and robustness. TextAttack is democratizing NLP: anyone can try data augmentation and adversarial training on any model or dataset, with just a few lines of code. Code and tutorials are available at this site.
- Abstract: An adversarial example is an input designed to fool a machine learning model. While there has been substantial research using adversarial attacks to analyze NLP models, each attack is implemented in its own code repository. It remains challenging to develop NLP attacks and utilize them to improve model performance. This paper introduces TextAttack, a Python framework for adversarial attacks, data augmentation, and adversarial training in NLP. TextAttack builds attacks from four components: a goal function, a set of constraints, a transformation, and a search method. TextAttack's modular design enables researchers to easily construct attacks from combinations of novel and existing components. TextAttack provides implementations of 16 adversarial attacks from the literature and supports a variety of models and datasets, including BERT and other transformers, and all GLUE tasks. TextAttack also includes data augmentation and adversarial training modules for using components of adversarial attacks to improve model accuracy and robustness. TextAttack is democratizing NLP: anyone can try data augmentation and adversarial training on any model or dataset, with just a few lines of code. Code and tutorials are available at this site.

### Our Github on TextAttack: `TextAttack <https://github.com/QData/TextAttack>`_
### Our Github on TextAttack: [https://github.com/QData/TextAttack](https://github.com/QData/TextAttack)

- Citations

Expand All @@ -29,7 +29,7 @@ How to Cite TextAttack

- Abstract: State-of-the-art attacks on NLP models lack a shared definition of a what constitutes a successful attack. We distill ideas from past work into a unified framework: a successful natural language adversarial example is a perturbation that fools the model and follows some linguistic constraints. We then analyze the outputs of two state-of-the-art synonym substitution attacks. We find that their perturbations often do not preserve semantics, and 38% introduce grammatical errors. Human surveys reveal that to successfully preserve semantics, we need to significantly increase the minimum cosine similarities between the embeddings of swapped words and between the sentence encodings of original and perturbed sentences.With constraints adjusted to better preserve semantics and grammaticality, the attack success rate drops by over 70 percentage points.

### Our Github on Reevaluation: `Reevaluating-NLP-Adversarial-Examples Github <https://github.com/QData/Reevaluating-NLP-Adversarial-Examples>`__
### Our Github on Reevaluation: [Reevaluating-NLP-Adversarial-Examples Github](https://github.com/QData/Reevaluating-NLP-Adversarial-Examples)

- Citations
```
Expand All @@ -49,7 +49,7 @@ How to Cite TextAttack

- Abstract: We study the behavior of several black-box search algorithms used for generating adversarial examples for natural language processing (NLP) tasks. We perform a fine-grained analysis of three elements relevant to search: search algorithm, search space, and search budget. When new search methods are proposed in past work, the attack search space is often modified alongside the search method. Without ablation studies benchmarking the search algorithm change with the search space held constant, an increase in attack success rate could from an improved search method or a less restrictive search space. Additionally, many previous studies fail to properly consider the search algorithms' run-time cost, which is essential for downstream tasks like adversarial training. Our experiments provide a reproducible benchmark of search algorithms across a variety of search spaces and query budgets to guide future research in adversarial NLP. Based on our experiments, we recommend greedy attacks with word importance ranking when under a time constraint or attacking long inputs, and either beam search or particle swarm optimization otherwise.

### Our Github on benchmarking: `TextAttack-Search-Benchmark Github <https://github.com/QData/TextAttack-Search-Benchmark>`__
### Our Github on benchmarking: [TextAttack-Search-Benchmark Github](https://github.com/QData/TextAttack-Search-Benchmark)


- Citations:
Expand Down