Skip to content
This repository was archived by the owner on Dec 18, 2024. It is now read-only.

Commit 4100336

Browse files
waylanwaylan
committed
Added --extension_configs option to the CLI.
The `--extension_configs` option must point to a YAML or JSON file. The contents of the file must parse to a Python Dict which will be passed to the `extension_configs` keyword of the `markdown.Markdown` class. Also added tests for all of the CLI option parsing options and updated documentation.
1 parent 726eacc commit 4100336

File tree

4 files changed

+251
-29
lines changed

4 files changed

+251
-29
lines changed

‎docs/cli.txt‎

Lines changed: 52 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -112,13 +112,59 @@ For a complete list of options, run
112112
Using Extensions
113113
----------------
114114

115-
For an extension to be run from the command line it must be provided in a module
116-
on your python path (see the [Extension API](extensions/api.html) for details).
117-
It can then be invoked by the name of that module:
115+
To load a Python-Markdown extension from the command line use the `-x`
116+
(or `--extension`) option. For extensions included with Python-Markdown, use
117+
the short "Name" [documented] for that extension.
118118

119-
$ markdown_py -x footnotes text_with_footnotes.txt > output.html
119+
[documented]: index.html#officially-supported-extensions
120120

121-
If the extension supports config options, you can pass them in as well:
121+
$ python -m markdown -x footnotes text_with_footnotes.txt
122122

123-
$ markdown_py -x "footnotes(PLACE_MARKER=~~~~~~~~)" input.txt
123+
For third party extensions, the extension module must be on your `PYTHONPATH`
124+
(see the [Extension API](extensions/api.html) for details). The extension can
125+
then be invoked by the name of that module using Python's dot syntax:
126+
127+
$ python -m markdown -x path.to.module input.txt
128+
129+
To load multiple extensions, specify an `-x` option for each extension:
130+
131+
$ python -m markdown -x footnotes -x codehilite input.txt
132+
133+
If the extension supports configuration options (see the documentation for the
134+
extension you are using to determine what settings it supports, if any), you
135+
can pass them in as well:
136+
137+
$ python -m markdown -x footnotes -c config.yml input.txt
138+
139+
The `-c` (or `--extension_configs`) option accepts a file name. The file must be in
140+
either the [YAML] or [JSON] format and contain YAML or JSON data that would map to
141+
a Python Dictionary in the format required by the [`extension_configs`][ec] keyword
142+
of the `markdown.Markdown` class. Therefore, the file `config.yaml` referenced in the
143+
above example might look like this:
144+
145+
footnotes:
146+
PLACE_MARKER: ~~~~~~~~
147+
UNIQUE_IDS: True
148+
149+
Note that while the `--extension_configs` option does specify the "footnotes" extension,
150+
you still need to load the extension with the `-x` option, or the configs for that
151+
extension will be ignored.
152+
153+
The `--extension_configs` option will only support YAML config files if [PyYaml] is
154+
installed on your system. JSON should work with no additional dependencies. The format
155+
of your config file is automatically detected.
156+
157+
As an alternative, you may append the extension configs as a string to the extension name
158+
that is provided to the `-x-` option in the following format:
159+
160+
$ python -m markdown -x "footnotes(PLACE_MARKER=~~~~~~~~,UNIQUE_IDS=1)" input.txt
161+
162+
Note that there are no quotes or whitespace in the above format, which severely limits
163+
how it can be used. For more complex settings, it is suggested that the
164+
`--extension_configs` option be used.
165+
166+
[ec]: reference.html#extension_configs
167+
[YAML]: http://yaml.org/
168+
[JSON]: http://json.org/
169+
[PyYAML]: http://pyyaml.org/
124170

‎docs/release-2.5.txt‎

Lines changed: 21 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -20,23 +20,34 @@ Backwards-incompatible Changes
2020
What's New in Python-Markdown 2.5
2121
---------------------------------
2222

23-
* The Extension Configuration code has been refactord to make it a little easier
24-
for extension authors to work with config settings. As a result, the
25-
[extension_configs] keyword now accepts a dictionary rather than requiring
26-
a list of tuples. A list of tuples is still supported so no one needs to change
27-
their existing code. This should simplify the learning curve for new users.
28-
29-
[extension_configs]: reference.html#extension_configs
30-
3123
* The [Smarty Extension] has had a number of additional configuration settings
3224
added, which allows one to define their own sustitutions to better support
3325
languages other than English. Thanks to [Martin Altmayer] for implementing this feature.
3426

3527
[Smarty Extension]: extensions/smarty.html
3628
[Martin Altmayer]:https://github.com/MartinAltmayer
3729

38-
There have been various refactors of the testing framework. While those changes
39-
will not directly effect end users, the code is being better tested whuch will
30+
* The Extension Configuration code has been refactord to make it a little easier
31+
for extension authors to work with config settings. As a result, the
32+
[`extension_configs`][ec] keyword now accepts a dictionary rather than requiring
33+
a list of tuples. A list of tuples is still supported so no one needs to change
34+
their existing code. This should also simplify the learning curve for new users.
35+
36+
[ec]: reference.html#extension_configs
37+
38+
* The [Command Line Interface][cli] now accepts a `--extensions_config` (or `-c`) option
39+
which accepts a filename and passes the parsed content of a [YAML] or [JSON] file to the
40+
[`extension_configs`][ec] keyword of the `markdown.Markdown` class. The conetents of
41+
the YAML or JSON must map to a Python Dictionary which matches the format required
42+
by the `extension_configs` kerword. Note that [PyYAML] is required to parse YAML files.
43+
44+
[cli]: cli.html#using-extensions
45+
[YAML]: http://yaml.org/
46+
[JSON]: http://json.org/
47+
[PyYAML]: http://pyyaml.org/
48+
49+
* There have been various refactors of the testing framework. While those changes
50+
will not directly effect end users, the code is being better tested which will
4051
benefit everyone.
4152

4253
* Various bug fixes have been made. See the

‎markdown/__main__.py‎

Lines changed: 37 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -7,13 +7,18 @@
77
import markdown
88
import sys
99
import optparse
10+
import codecs
11+
try:
12+
import yaml
13+
except ImportError:
14+
import json as yaml
1015

1116
import logging
1217
from logging import DEBUG, INFO, CRITICAL
1318

1419
logger = logging.getLogger('MARKDOWN')
1520

16-
def parse_options():
21+
def parse_options(args=None, values=None):
1722
"""
1823
Define and parse `optparse` options for command-line usage.
1924
"""
@@ -29,28 +34,36 @@ def parse_options():
2934
metavar="OUTPUT_FILE")
3035
parser.add_option("-e", "--encoding", dest="encoding",
3136
help="Encoding for input and output files.",)
32-
parser.add_option("-q", "--quiet", default = CRITICAL,
33-
action="store_const", const=CRITICAL+10, dest="verbose",
34-
help="Suppress all warnings.")
35-
parser.add_option("-v", "--verbose",
36-
action="store_const", const=INFO, dest="verbose",
37-
help="Print all warnings.")
3837
parser.add_option("-s", "--safe", dest="safe", default=False,
3938
metavar="SAFE_MODE",
4039
help="'replace', 'remove' or 'escape' HTML tags in input")
4140
parser.add_option("-o", "--output_format", dest="output_format",
4241
default='xhtml1', metavar="OUTPUT_FORMAT",
4342
help="'xhtml1' (default), 'html4' or 'html5'.")
44-
parser.add_option("--noisy",
45-
action="store_const", const=DEBUG, dest="verbose",
46-
help="Print debug messages.")
47-
parser.add_option("-x", "--extension", action="append", dest="extensions",
48-
help = "Load extension EXTENSION.", metavar="EXTENSION")
4943
parser.add_option("-n", "--no_lazy_ol", dest="lazy_ol",
5044
action='store_false', default=True,
5145
help="Observe number of first item of ordered lists.")
46+
parser.add_option("-x", "--extension", action="append", dest="extensions",
47+
help = "Load extension EXTENSION.", metavar="EXTENSION")
48+
parser.add_option("-c", "--extension_configs", dest="configfile", default=None,
49+
help="Read extension configurations from CONFIG_FILE. "
50+
"CONFIG_FILE must be of JSON or YAML format. YAML format requires "
51+
"that a python YAML library be installed. The parsed JSON or YAML "
52+
"must result in a python dictionary which would be accepted by the "
53+
"'extension_configs' keyword on the markdown.Markdown class. "
54+
"The extensions must also be loaded with the `--extension` option.",
55+
metavar="CONFIG_FILE")
56+
parser.add_option("-q", "--quiet", default = CRITICAL,
57+
action="store_const", const=CRITICAL+10, dest="verbose",
58+
help="Suppress all warnings.")
59+
parser.add_option("-v", "--verbose",
60+
action="store_const", const=INFO, dest="verbose",
61+
help="Print all warnings.")
62+
parser.add_option("--noisy",
63+
action="store_const", const=DEBUG, dest="verbose",
64+
help="Print debug messages.")
5265

53-
(options, args) = parser.parse_args()
66+
(options, args) = parser.parse_args(args, values)
5467

5568
if len(args) == 0:
5669
input_file = None
@@ -60,10 +73,21 @@ def parse_options():
6073
if not options.extensions:
6174
options.extensions = []
6275

76+
extension_configs = {}
77+
if options.configfile:
78+
with codecs.open(options.configfile, mode="r", encoding=options.encoding) as fp:
79+
try:
80+
extension_configs = yaml.load(fp)
81+
except yaml.YAMLError as e:
82+
message = "Failed parsing extension config file: %s" % options.configfile
83+
e.args = (message,) + e.args[1:]
84+
raise
85+
6386
return {'input': input_file,
6487
'output': options.filename,
6588
'safe_mode': options.safe,
6689
'extensions': options.extensions,
90+
'extension_configs': extension_configs,
6791
'encoding': options.encoding,
6892
'output_format': options.output_format,
6993
'lazy_ol': options.lazy_ol}, options.verbose

‎tests/test_apis.py‎

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,9 +10,14 @@
1010
from __future__ import unicode_literals
1111
import unittest
1212
import sys
13+
import os
1314
import types
1415
import markdown
1516
import warnings
17+
from markdown.__main__ import parse_options
18+
from logging import DEBUG, INFO, CRITICAL
19+
import yaml
20+
import tempfile
1621

1722
PY3 = sys.version_info[0] == 3
1823

@@ -433,3 +438,139 @@ def testBooleansParsing(self):
433438

434439
def testInvalidBooleansParsing(self):
435440
self.assertRaises(ValueError, markdown.util.parseBoolValue, 'novalue')
441+
442+
class TestCliOptionParsing(unittest.TestCase):
443+
""" Test parsing of Command Line Interface Options. """
444+
445+
def setUp(self):
446+
self.default_options = {
447+
'input': None,
448+
'output': None,
449+
'encoding': None,
450+
'safe_mode': False,
451+
'output_format': 'xhtml1',
452+
'lazy_ol': True,
453+
'extensions': [],
454+
'extension_configs': {},
455+
}
456+
self.tempfile = ''
457+
458+
def tearDown(self):
459+
if os.path.isfile(self.tempfile):
460+
os.remove(self.tempfile)
461+
462+
def testNoOptions(self):
463+
options, logging_level = parse_options([])
464+
self.assertEqual(options, self.default_options)
465+
self.assertEqual(logging_level, CRITICAL)
466+
467+
def testQuietOption(self):
468+
options, logging_level = parse_options(['-q'])
469+
self.assertTrue(logging_level > CRITICAL)
470+
471+
def testVerboseOption(self):
472+
options, logging_level = parse_options(['-v'])
473+
self.assertEqual(logging_level, INFO)
474+
475+
def testNoisyOption(self):
476+
options, logging_level = parse_options(['--noisy'])
477+
self.assertEqual(logging_level, DEBUG)
478+
479+
def testInputFileOption(self):
480+
options, logging_level = parse_options(['foo.txt'])
481+
self.default_options['input'] = 'foo.txt'
482+
self.assertEqual(options, self.default_options)
483+
484+
def testOutputFileOption(self):
485+
options, logging_level = parse_options(['-f', 'foo.html'])
486+
self.default_options['output'] = 'foo.html'
487+
self.assertEqual(options, self.default_options)
488+
489+
def testInputAndOutputFileOptions(self):
490+
options, logging_level = parse_options(['-f', 'foo.html', 'foo.txt'])
491+
self.default_options['output'] = 'foo.html'
492+
self.default_options['input'] = 'foo.txt'
493+
self.assertEqual(options, self.default_options)
494+
495+
def testEncodingOption(self):
496+
options, logging_level = parse_options(['-e', 'utf-8'])
497+
self.default_options['encoding'] = 'utf-8'
498+
self.assertEqual(options, self.default_options)
499+
500+
def testSafeModeOption(self):
501+
options, logging_level = parse_options(['-s', 'escape'])
502+
self.default_options['safe_mode'] = 'escape'
503+
self.assertEqual(options, self.default_options)
504+
505+
def testOutputFormatOption(self):
506+
options, logging_level = parse_options(['-o', 'html5'])
507+
self.default_options['output_format'] = 'html5'
508+
self.assertEqual(options, self.default_options)
509+
510+
def testNoLazyOlOption(self):
511+
options, logging_level = parse_options(['-n'])
512+
self.default_options['lazy_ol'] = False
513+
self.assertEqual(options, self.default_options)
514+
515+
def testExtensionOption(self):
516+
options, logging_level = parse_options(['-x', 'footnotes'])
517+
self.default_options['extensions'] = ['footnotes']
518+
self.assertEqual(options, self.default_options)
519+
520+
def testMultipleExtensionOptions(self):
521+
options, logging_level = parse_options(['-x', 'footnotes', '-x', 'smarty'])
522+
self.default_options['extensions'] = ['footnotes', 'smarty']
523+
self.assertEqual(options, self.default_options)
524+
525+
def create_config_file(self, config):
526+
""" Helper to create temp config files. """
527+
if not isinstance(config, markdown.util.string_type):
528+
# convert to string
529+
config = yaml.dump(config)
530+
fd, self.tempfile = tempfile.mkstemp('.yml')
531+
with os.fdopen(fd, 'w') as fp:
532+
fp.write(config)
533+
534+
def testExtensonConfigOption(self):
535+
config = {
536+
'wikilinks': {
537+
'base_url': 'http://example.com/',
538+
'end_url': '.html',
539+
'html_class': 'test',
540+
},
541+
'footnotes': {
542+
'PLACE_MARKER': '~~~footnotes~~~'
543+
}
544+
}
545+
self.create_config_file(config)
546+
options, logging_level = parse_options(['-c', self.tempfile])
547+
self.default_options['extension_configs'] = config
548+
self.assertEqual(options, self.default_options)
549+
550+
def testExtensonConfigOptionAsJSON(self):
551+
config = {
552+
'wikilinks': {
553+
'base_url': 'http://example.com/',
554+
'end_url': '.html',
555+
'html_class': 'test',
556+
},
557+
'footnotes': {
558+
'PLACE_MARKER': '~~~footnotes~~~'
559+
}
560+
}
561+
import json
562+
self.create_config_file(json.dumps(config))
563+
options, logging_level = parse_options(['-c', self.tempfile])
564+
self.default_options['extension_configs'] = config
565+
self.assertEqual(options, self.default_options)
566+
567+
def testExtensonConfigOptionMissingFile(self):
568+
self.assertRaises(IOError, parse_options, ['-c', 'missing_file.yaml'])
569+
570+
def testExtensonConfigOptionBadFormat(self):
571+
config = """
572+
[footnotes]
573+
PLACE_MARKER= ~~~footnotes~~~
574+
"""
575+
self.create_config_file(config)
576+
self.assertRaises(yaml.YAMLError, parse_options, ['-c', self.tempfile])

0 commit comments

Comments
 (0)