Python - Best practice for Python main function definition and program start/exit

exit()

The exit() will straightforwardly end the execution of the Python code/script. The code that follows thereafter the exit() command won’t be executed.

You can think of exit() as an alias for quit() (or vice-versa) in Python. They simply co-exist to make Python more user-friendly. However, it is not a good practice to use quit() and exit() in production code and should only be reserved for use in the interpreter.

sys.exit(main())

This will call the function main() and when main finishes, it will exit giving the system the return code that is the result of main().

A simplified example where this might be used:

def main():
    try:
        doSomething()
        return 0
    except:
        return 1


if __name__ == "__main__":
    sys.exit(main())

exit() vs. sys.exit()

import sys
def foo():
    try:
        print("Demonstrating sys.exit()")
        sys.exit()
        print("This line will not be executed!")
    except SystemExit:
        # argument denoting exit status
        print("Abnormal Termination! Encountered SystemExit")
def func():
    print("Demonstrating exit()")
    print(exit)
    exit()  # lines after this statement are ignored and are not executed
    print("The previous command executes ignoring all the lines after the command")
# Calling both the function:
foo()
print()
func()

Result:

Demonstrating sys.exit()
Abnormal Termination! Encountered SystemExit

Demonstrating exit()
Use exit() or Ctrl-D (i.e. EOF) to exit

When you need something more fancy working with sys.exit() explicitly is a good idea. That’s basically your future approach. But don’t only think of scripts called directly, also think of installed packages. Setuptools has a cross-platform mechanism to define functions as entry points for scripts. If you have this in your setup.py:

from setuptools import setup

setup(
    name='scraper',
    version='1.0.0',
    install_requires=[
        'boto3>=1.17.17',
        'keyring>=22.3.0',
        'bs4',
        'requests_ntlm'
    ],
    entry_points={
        'console_scripts': [
            'scraper=scraper.__main__:main'
        ]
    },
)

and install that package you can run your_script from the command line.

pip install -e ./

scraper  -c conf/conf_upload.json 

Template of code structure would be:

import sys
from argparse import ArgumentParser, Namespace
from typing import Dict, List
import yaml  # just used as an example here for loading more configs, optional


def parse_arguments(cli_args: List[str] = None) -> Namespace:
    parser = ArgumentParser()
    # parser.add_argument()
    # ...
    return parser.parse_args(args=cli_args)  # None defaults to sys.argv[1:]


def load_configs(args: Namespace) -> Dict:
    try:
        with open(args.config_path, 'r') as file_pointer:
            config = yaml.safe_load(file_pointer)

        # arrange and check configs here

        return config
    except Exception as err:
        # log errors
        print(err)
        if err == "Really Bad":
            raise err

        # potentionally return some sane fallback defaults if desired/reasonable
        sane_defaults = []
        return sane_defaults


def main(args: Namespace = parse_arguments()) -> int:
    try:
        # maybe load some additional config files here or in a function called here
        # e.g. args contains a path to a config folder; or use sane defaults
        # if the config files are missing(if that is your desired behavior)
        config = load_configs(args)
        do_real_work(args, config)

    except KeyboardInterrupt:
        print("Aborted manually.", file=sys.stderr)
        return 1

    except Exception as err:
        # (in real code the `except` would probably be less broad)
        # Turn exceptions into appropriate logs and/or console output.

        # log err
        print("An unhandled exception crashed the application!", err)

        # non-zero return code to signal error
        # Can of course be more fine grained than this general
        # "something went wrong" code.
        return 1

    return 0  # success


# __main__ support is still here to make this file executable without
# installing the package first.
if __name__ == "__main__":
    sys.exit(main(parse_arguments()))

Another example,

import sys
import importlib
import argparse
import json


def main():
    conf_default_path = '{:}/conf/conf_upload.json'.format(os.path.abspath(__file__ + "/../../"))
    parser = argparse.ArgumentParser(description='Transform files from/to an AWS-S3 object.')
    parser.add_argument("-v", "--verbose",
                        help="increase output verbosity",
                        action="store_true")
    parser.add_argument("-c", "--conf-path",
                        help="path to pipeline json config file",
                        default=conf_default_path,
                        type=str, )

    args = parser.parse_args()
    if not os.path.exists(args.conf_path):
        raise IOError('Pipeline JSON config file {:} does not exist!'.format(args.conf_path))

    with open(args.conf_path) as f:
        conf = json.load(f)

    for stage in conf['pipeline']:
        # import module
        module = importlib.import_module('scraper.' + stage['module'])

        # run module
        module.run(**stage['params'])


if __name__ == '__main__':
    sys.exit(main())