You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
commonast.py is a static analysis tool to count different programming language constructs.
8
+
The tool functions in two modes: **count mode** and **print mode**
9
+
10
+
### Count Mode
11
+
12
+
```
13
+
python commonast.py lang nodeType arg filename1 filename2 .... filenamen
14
+
```
15
+
will output the number of ```nodeType```s under the argument ```arg``` in the source file ```filename``` which is written in language ```lang```. It will also output valuable information about the AST depending on the outputOption.
16
+
17
+
Supported ```lang```s:
18
+
* -py
19
+
* -cpp
20
+
21
+
22
+
Supported ```nodeType```s:
23
+
* -For counts the number of for loops
24
+
* Supported args:
25
+
* -Void
26
+
27
+
* -While counts the number of while loops
28
+
* Supported args:
29
+
* -Void
30
+
31
+
* -Call counts the number of calls (of a certain name or just calls in general)
32
+
* Supported args:
33
+
* -Void
34
+
* Specific call name that we want to count (example: only count the number of calls to print)
35
+
36
+
(The infrastructure is there to count many more nodes. I'm only adding them to this once they've been tested against Sam's tool to make sure they're correct)
37
+
38
+
You can count nodes on any number of filenames. The number returned will be the sum number of nodeTypes in all of the filenames.
An interactive view of the data can be produced by specifying HTML format:
59
+
60
+
```
61
+
/usr/local/submitty/SubmittyAnalysisTools/diagnostics -l python --format html file.py
62
+
```
63
+
64
+
For example, if you would like to count additions, but are unsure of which token to count, you could use a test file like:
65
+
66
+
```
67
+
# file.py
68
+
print(1 + 1)
69
+
```
70
+
71
+
Running `/usr/local/submitty/SubmittyAnalysisTools/diagnostics -l python file.py` on this file will produce the following output:
72
+
73
+
```
74
+
{
75
+
"/absolute/path/to/file.py": {
76
+
"tokens": [
77
+
{
78
+
"end_col": 6,
79
+
"token": "Identifier",
80
+
"start_line": 2,
81
+
"start_col": 1,
82
+
"end_line": 2
83
+
},
84
+
{
85
+
"end_col": 7,
86
+
"token": "LeftParen",
87
+
"start_line": 2,
88
+
"start_col": 6,
89
+
"end_line": 2
90
+
},
91
+
{
92
+
"end_col": 8,
93
+
"token": "IntegerLiteral",
94
+
"start_line": 2,
95
+
"start_col": 7,
96
+
"end_line": 2
97
+
},
98
+
{
99
+
"end_col": 10,
100
+
"token": "Plus",
101
+
"start_line": 2,
102
+
"start_col": 9,
103
+
"end_line": 2
104
+
},
105
+
{
106
+
"end_col": 12,
107
+
"token": "IntegerLiteral",
108
+
"start_line": 2,
109
+
"start_col": 11,
110
+
"end_line": 2
111
+
},
112
+
{
113
+
"end_col": 13,
114
+
"token": "RightParen",
115
+
"start_line": 2,
116
+
"start_col": 12,
117
+
"end_line": 2
118
+
}
119
+
],
120
+
"nodes" : { ... node data here ... }
121
+
}
122
+
}
123
+
```
124
+
125
+
The ``token`` fields specify tokens that can be given to ``submitty_count``.
126
+
Notice that a token ``Plus`` is present between two ``IntegerLiteral`` tokens.
127
+
You could verify that this is the right token by looking at the ``start_line``, ``end_line``, ``start_col``, and ``end_col`` fields, which indicate on what row and column the tokens begin and end within the file.
128
+
Once you are sure that the token is correct, you could count it within student submissions with ``submitty_count``:
129
+
130
+
```
131
+
submitty_count -l python *.py
132
+
```
52
133
53
134
### Tokens
54
135
@@ -107,12 +188,6 @@ should be the first tool considered when writing an assignment that
107
188
requires static analysis. Only seek out more advanced options when
108
189
necessary.
109
190
110
-
__TODO: Insert instructions to produce the intermediate tokens so the
111
-
instructor user can experiment.__
112
-
113
-
__TODO: Insert link to list of valid tokens that can counted.__
114
-
115
-
116
191
### Nodes
117
192
118
193
The next level of analysis enables counting _nodes_ within a parse tree, which
@@ -146,15 +221,15 @@ what kind of literal is present. This enables the counting of
146
221
specific classes of node. For example:
147
222
148
223
```
149
-
"submitty_count -l python literal *.py"
224
+
submitty_count -l python literal *.py
150
225
```
151
226
152
227
If run upon the code fragment from the start of this section, this will yield 3,
153
228
counting all literals used within the code. Contrast:
154
229
which will return `3`. In contrast:
155
230
156
231
```
157
-
"submitty_count -l python integer *.py"
232
+
submitty_count -l python integer *.py
158
233
```
159
234
160
235
will return `2`, as it will only count the integer literals.
@@ -169,12 +244,6 @@ counting approach. However, these features have different nodes in the parse
169
244
tree, so by counting nodes with certain tags it is possible to easily
170
245
distinguish them.
171
246
172
-
__TODO: Insert instructions to produce a human readable version of the
173
-
parse tree so the instructor user can experiment.__
174
-
175
-
__TODO: Insert link to valid tags (& nodes?) that can counted.__
176
-
177
-
178
247
### Function Calls
179
248
180
249
This method is a bit higher level: it attempts via a language-dependent method
@@ -185,9 +254,5 @@ method at RPI is determining the number of calls to the ``print`` function
185
254
present in Python code, for example:
186
255
187
256
```
188
-
"submitty_count call print -l py *.py"
257
+
submitty_count call print -l py *.py
189
258
```
190
-
191
-
__TODO: Insert instructions to produce a human readable version of the
192
-
functions found in a specific program(?) so the instructor user can experiment.__
0 commit comments