src/bin/sage: Delegate handling of options of sage-the-distribution, specifying a subset that is supported by sagelib proper

[mkoeppe]

The sage script has a number of options that only make sense for sage-the-distribution, but do not make sense in downstream packaging of sage. Consequently, downstream packagers may install simple custom versions of the script that only provide a subset of the options.

Examples:

• sage -sqlite -- as discussed in spkg-configure.m4 for sqlite #29002/sqlite3 CLI executable should be checked for at configure time #29092 -- because from a downstream packaging point of view, sage is not really responsible for providing sqlite, it's provided by justs another system package
• sage -upgrade -- only makes sense with sage-the-distribution built from source

We should specify the subset of sage command line options that must be supported in any deployment/packaging of sage. This is so that user packages can work reliably.

Examples:

• sage -c SAGECOMMAND -- obviously
• sage -python -c COMMAND
• sage -t -- because it is used by user packages for testing sage sources
• sage -sh ..... ?

In this ticket, we actually split src/bin/sage into sagelib functionality and sage-the-distribution functionality. src/bin/sage delegates unknown options to build/bin/sage-site.

Ideally, distributions should be able to use an unmodified src/bin/sage.

Context:

• https://groups.google.com/d/msg/sage-packaging/BmkxIBdwbvE/fRMl2sjdBQAJ
• Meta-ticket: Add Dockerfiles and CI scripts for integration testing of source and binary distributions and of downstream packages #29060: Meta-ticket: Add Dockerfiles and CI scripts for integration testing of source and binary distributions and of downstream packages

See also:

• src/doc/bootstrap: Generate src/doc/en/reference/repl/options.txt #29884: src/doc/bootstrap: Generate src/doc/en/reference/repl/options.rst

Depends on #29878
Depends on #29289
Depends on #29884

CC: @kiwifb @isuruf @dimpase @embray @saraedum @antonio-rojas @slel @EmmanuelCharpentier @orlitzky @kcrisman

Component: scripts

Author: Matthias Koeppe, John Palmieri

Branch: 3953671

Reviewer: Matthias Koeppe, John Palmieri, François Bissey

Issue created by migration from https://trac.sagemath.org/ticket/29111

[mkoeppe]

comment:1

As I said on #29092, we should also move corresponding sage-the-distribution tests (such as those for sage -sqlite) mentioned in #29002) from src (which is for sagelib) to somewhere in build (which is for sage-the-distribution). Downstream packagers would ignore the sage-the-distribution tests.

[mkoeppe]

comment:3

pushing these forward to 9.2

[mkoeppe]

Description changed:

--- 
+++ 
@@ -12,7 +12,7 @@
 - `sage -t` -- because it is used by user packages for testing sage sources
 - `sage -sh` ..... ?
 
-(In addition to specifying this, we might actually want to split `src/bin/sage` into sagelib functionality and sage-the-distribution functionality.)
+(In addition to specifying this, we might actually want to split `src/bin/sage` into sagelib functionality and sage-the-distribution functionality. Ideally, distributions should be able to use an unmodified `src/bin/sage`, which would delegate unknown options to an optional "site" script that would be provided by sage-the-distribution.)
 
 Context:
 - https://groups.google.com/d/msg/sage-packaging/BmkxIBdwbvE/fRMl2sjdBQAJ

[jhpalmieri]

comment:6

I've asked this question before: if Sage is built with the system's Python, should we still keep sage --python ...? What is the difference between sage --python and sage --sqlite3 (and what about sage --gap, sage --singular, etc.)?

[jhpalmieri]

comment:7

Here are some arguments that should always be allowed:

• sage -t
• sage -c
• sage -h
• sage -n, sage --notebook
• sage --nodotsage
• sage -v, sage --version

"Advanced" arguments:

• sage --preparse
• sage --min
• sage -q
• sage --dumpversion
• sage --startuptime
• sage --fixdoctests
• sage --coverage
• sage --coverageall
• sage --grep, sage --search_src, sage --grepdoc, sage --search_doc
• sage --rst2ipynb, sage --ipynb2rst

Maybe also sage --sh.

(I'm not saying is a complete list, but I would suggest these as a starting point.)

[jhpalmieri]

comment:8

By the way, sage -twistd doesn't work: there is no twistd executable since Sage 9.1. Plus we are removing the twisted package in #29754. See #29878.

[kiwifb]

comment:9

Hum, this ticket reminds me that I need to clean the version in sage-on-gentoo

fbissey@moonloop ~ $ sage -advanced
SageMath version 9.2.beta1, Release Date: 2020-06-13

Running Sage:
  file.[sage|py|spyx] -- run given .sage, .py or .spyx file
  -advanced           -- print this advanced help message
  -c <cmd>            -- Evaluates cmd as sage code
  -h, -?              -- print short help message
  -min [...]          -- do not populate global namespace (must be first
                         option)
  -preparse <file.sage> -- preparse file.sage and produce corresponding file.sage.py
  -q                  -- quiet; start with no banner
  -gthread, -qthread, -q4thread, -wthread, -pylab
                      -- pass the option through to ipython
  -v, -version        -- display Sage version information
  -dumpversion        -- print Sage version

Running the notebook:
  --notebook=[...]    -- start the Sage notebook (valid options are
                         'default', 'sagenb', 'jupyter' and 'export').
                         Current default is 'export' sagenb to jupyter.
                         See the output of sage --notebook --help
                         for more details and examples of how to pass
                         optional arguments
  -inotebook [...]    -- start the *insecure* Sage notebook (deprecated)
  -n, -notebook [...] -- start the default Sage notebook (options are the
                         same as for the notebook command in Sage).  See the
                         output of sage -n -h for more details

Running external programs:
  -cleaner            -- run the Sage cleaner
  -cython [...]       -- run Cython with given arguments
  -ecl [...]          -- run Common Lisp
  -gap [...]          -- run Sage's Gap with given arguments
  -gdb                -- run Sage under the control of gdb
  -gp [...]           -- run Sage's PARI/GP calculator with given arguments
  -ipython [...]      -- run Sage's IPython using the default environment (not
                         Sage), passing additional options to IPython
  -ipython3 [...]     -- same as above, but using Python 3
  -jupyter [...]      -- run Sage's Jupyter with given arguments
  -kash [...]         -- run Sage's Kash with given arguments
                         (not installed currently, run sage -i kash)
  -lisp [...]         -- run Lisp interpreter included with Sage
  -M2 [...]           -- run Sage's Macaulay2 with given arguments
                         (not installed currently, run sage -i macaulay2)
  -maxima [...]       -- run Sage's Maxima with given arguments
  -mwrank [...]       -- run Sage's mwrank with given arguments
  -polymake [...]     -- run Sage's Polymake with given arguments
                         (not installed currently, run sage -i polymake)
  -python [...]       -- run the Python interpreter
  -python2 [...]      -- run the Python 2 interpreter
  -python3 [...]      -- run the Python 3 interpreter
  -R [...]            -- run Sage's R with given arguments
  -sh [...]           -- run $SHELL (/bin/bash) with Sage environment variables
  -singular [...]     -- run Sage's singular with given arguments
  -sqlite3 [...]      -- run Sage's sqlite3 with given arguments
  -twistd [...]       -- run Twisted server

Testing the Sage library:
  -startuptime [module] -- display how long each component of Sage takes to
                         start up; optionally specify a module to get more
                         details about that particular module
  -t [options] <--all|files|dir>
                      -- test examples in .py, .pyx, .sage, .tex or .rst files
                         selected options:
                           --long - include lines with the phrase 'long time'
                           --verbose - print debugging output during the test
                           --all - test all files
                           --sagenb - test all sagenb files
                           --optional - controls which optional tests are run
                           --initial - only show the first failure per block
                           --debug - drop into PDB after an unexpected error
                           --failed - only test files that failed last test
                           --warn-long [timeout] - warning if doctest is slow
                           --only-errors - only output failures, not successes
                           --gc=GC - control garbarge collection (ALWAYS:
                                     collect garbage before every test; NEVER:
                                     disable gc; DEFAULT: Python default)
                           --help - show all testing options
  -tp <N> [...]       -- like -t above, but tests in parallel using N threads
                         with 0 interpreted as a sensible default
  -testall [options]  -- test all source files, docs, and examples.  options
                         like -t

File conversion:
  -rst2ipynb [...]    -- Generates Jupyter notebook (.ipynb) from standalone
                         reStructuredText source.
                         (not installed currently, run sage -i rst2ipynb)
  -ipynb2rst [...]    -- Generates a reStructuredText source file from
                         a Jupyter notebook (.ipynb).

Valgrind memory debugging:
  -cachegrind         -- run Sage using Valgrind's cachegrind tool.  The log
                         files are named sage-cachegrind.PID can be found in
                         
  -callgrind          -- run Sage using Valgrind's callgrind tool.  The log
                         files are named sage-callgrind.PID can be found in
                         
  -massif             -- run Sage using Valgrind's massif tool.  The log
                         files are named sage-massif.PID can be found in
                         
  -memcheck           -- run Sage using Valgrind's memcheck tool.  The log
                         files are named sage-memcheck.PID can be found in
                         
  -omega              -- run Sage using Valgrind's omega tool.  The log
                         files are named sage-omega.PID can be found in
                         
  -valgrind           -- this is an alias for -memcheck

You can also use -- before a long option, e.g., 'sage --optional'.

I have to drop the py2/py3 stuff for just defaut python. I dropped -sh the day I dropped sage-env altogether, the main point of keeping sage -sh around was to check or use the environment set by sage-env.

[mkoeppe]

comment:10

Dropping sage -sh is a bad idea. User packages need this

[mkoeppe]

comment:11

Replying to @jhpalmieri:

I've asked this question before: if Sage is built with the system's Python, should we still keep sage --python ...? What is the difference between sage --python and sage --sqlite3 (and what about sage --gap, sage --singular, etc.)?

sage -python means: run the python that sage runs in.
In distributions, this could just be plain Python.

Install scripts of user packages need this.

[mkoeppe]

Dependencies: #29878

[kiwifb]

comment:13

Replying to @mkoeppe:

Dropping sage -sh is a bad idea. User packages need this

Certainly, vanilla sage does need it. In sage-on-gentoo at best you would have a normal shell since I dropped sage-env altogether.

[mkoeppe]

comment:14

Replying to @kiwifb:

Replying to @mkoeppe:

Dropping sage -sh is a bad idea. User packages need this

Certainly, vanilla sage does need it. In sage-on-gentoo at best you would have a normal shell since I dropped sage-env altogether.

Yes, vanilla shell is fine. The point is that user packages need the interface.

[mkoeppe]

Changed dependencies from #29878 to #29878, #29289

[kiwifb]

comment:16

Now, I understand what you mean. User scripts may start with sage -sh and what not. I completely forgot about that when I removed it - even though that was one of the reason I was keeping it around. I will re-instate it as a vanilla shell or no-op for compatibility.

[mkoeppe]

comment:17

Great!