python module notes

Each OS/Linux distribution has a combination of different install systems including apt, rpm, pip, conda, homebrew. Each install system consists of a central repo to distribute packages and a format for a config file to run both system OS commands, shell scripts and code; where code is some form of standard programming language for additional flexibility. 

There are multiple ways to install python programs/modules, apt install, pip install, conda install, brew install. 

After installation the expectation is import xxx can be used and a CLI if there is one. 

sys.path + PYTHON_PATH environment variables for imp module to look for package_name using import package_name. imp.find_module('torch') to find the module path. 

pip install: installs to directory prefix/site-packages

apt install: installs to directory prefix/dist-packages

distutils was the original python2.x specification using setup.py to install to a site-packages directory. This has been replaced by distutils2 which has been replaced by pip. The original convention of setup.py and site-packages still holds but has been supplemented by setuptools. Setuptools allows a python function specified by module path, module file and function name to be the executable command.

virtual environment site-packages:like MACOS and everybody else there is a semi global path for the anaconda env and one for the user level installed venvs in another scope. 

/anaconda3/lib/pythonX/.../site-packages

/anaconda3/venv/pythonX/.../site-packages

MACOS site-packages: global python install /Libraray/Python/2.7/site-packages; needed for MACOS python system functionality. 

MACOS site-packages: another python install at /usr/local/Cellar/xxx/site-packages for homebrew. User installed packages

To package a python module, we specify an executable name for the program and a function which is run when the executable is typed in the command line. For example foo would run function main under file name foo_file under directory foo_module would have 'console_scripts':['foo=foo_module.foo_file.main']

Use python modules setuptools and click. The setuptools entry_points parameter is a user configurable CLI.

entry_points={
'console_scripts':['executable_name=directory.module_file.py name.function_name']
}


Not all executable names work, aaa has to be an available name and not used for another program. You may not get an error message in the case of conflict. find_packages can be used instead of manually specifying packages which is a list of all the directories you want to include:

from setuptools import setup, find_packages

setup(

name='test_py',

version='0.1.0',

packages=['pt'],

install_requires=[

'Click',

],

entry_points={

'console_scripts': [

'aaa = pt.file_name:main',

],

},

)

pt
  setup.py
  pt
  __init__.py
  file_name.py


>pip install --editable . in the directory where setup.py is located

@click.option, @click.pass_context,

Train a model:
Predict a data sample: