YouTip

# Python3 Modules \n\n## Python3.x Python3 Modules\n\nIn the previous chapters, we primarily used the Python interpreter for programming. If you exit the Python interpreter and re-enter it, all the methods and variables you defined will be lost.\n\nTo address this, Python provides a way to store these definitions in a file for use by scripts or interactive interpreter instances. This file is called a **module**.\n\nA module in Python is a file containing Python definitions and statements. The filename is the module name with the `.py` suffix appended.\n\nModules can contain functions, classes, variables, and executable code. Through modules, we can organize code into reusable units, making it easier to manage and maintain.\n\n### Purpose of Modules\n\n* **Code Reusability**: Encapsulate commonly used functions into modules for repeated use across multiple programs.\n* **Namespace Management**: Modules help avoid naming conflicts. Functions or variables with the same name in different modules do not interfere with each other.\n* **Code Organization**: Divide code into different modules based on functionality, making the program structure clearer.\n\nHere is an example of using a module from the Python standard library.\n\n## Example (Python 3.0+)\n\n```python\nimport sys\n\nprint('Command line arguments are:')\nfor i in sys.argv:\n print(i)\n\nprint('nnPython path is:', sys.path, 'n')\n\nThe execution result is as follows:\n\n$ python using_sys.py Parameter 1 Parameter 2\nCommand line arguments are:\nusing_sys.py\nParameter 1\nParameter 2\n\nPython path is: ['/root', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/usr/lib/python3.4/lib-dynload', '/usr/local/lib/python3.4/dist-packages', '/usr/lib/python3/dist-packages']\n\n* 1. `import sys` imports the `sys.py` module from the Python standard library; this is the way to import a module.\n* 2. `sys.argv` is a list containing command-line arguments.\n* 3. `sys.path` contains a list of paths where the Python interpreter automatically searches for required modules.\n\n* * *\n\n## The import Statement\n\nTo use a Python source file, you simply execute the `import` statement in another source file. The syntax is:\n\n```python\nimport module1[, module2[, ... moduleN]]\n\nWhen the interpreter encounters an `import` statement, it imports the module if it is in the current search path.\n\nThe search path is a list of all directories that the interpreter searches first. To import a module named `support`, you need to place the command at the top of the script.\n\n## support.py File Code\n\n```python\ndef print_func(par):\n print("Hello : ", par)\n return\n\ntest.py imports the support module:\n\n## test.py File Code\n\n```python\nimport support\n\nsupport.print_func("")\n\nThe output of the above example is:\n\n$ python3 test.py\nHello : \n\n(\n\nA module is only imported once, no matter how many times you execute `import`. This prevents the imported module from being executed repeatedly.\n\nWhen we use the `import` statement, how does the Python interpreter find the corresponding file?\n\nThis involves Python's search path, which is a list of directory names. The Python interpreter searches for the imported module in these directories sequentially.\n\n### Module Search Path\n\nWhen importing a module, Python searches for it in the following order:\n\n1. The current directory.\n2. Directories specified by the environment variable `PYTHONPATH`.\n3. The Python standard library directory.\n4. Directories specified in `.pth` files.\n\nThe search path is determined during Python compilation or installation. Installing new libraries may also modify it. The search path is stored in the `path` variable within the `sys` module. A simple experiment in the interactive interpreter:\n\n```python\n>>> import sys\n>>> sys.path\n['', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/usr/lib/python3.4/lib-dynload', '/usr/local/lib/python3.4/dist-packages', '/usr/lib/python3/dist-packages']\n>>>\n\nThe output of `sys.path` is a list, where the first item is an empty string `''`, representing the current directory (if printed from a script, it's clearer which directory it is), i.e., the directory where we execute the Python interpreter (for scripts, it's the directory where the running script is located).\n\nTherefore, if there is a file with the same name as the module to be imported in the current directory, it will shadow the module to be imported.\n\nUnderstanding the concept of the search path allows you to modify `sys.path` in a script to import modules not in the search path.\n\nNow, create a `fibo.py` file in the current directory of the interpreter or in a directory within `sys.path`, with the following code:\n\n## Example\n\n```python\ndef fib(n):\n a, b = 0, 1\n while b < n:\n print(b, end='')\n a, b = b, a+b\n print()\n\ndef fib2(n):\n result = []\n a, b = 0, 1\n while b >> import fibo\n\nThis does not write the function names defined directly in `fibo` into the current symbol table; it only writes the module name `fibo` there.\n\nYou can access functions using the module name:\n\n## Example\n\n```python\n>>> fibo.fib(1000)\n1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987\n>>> fibo.fib2(100)\n[1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]\n>>> fibo.__name__\n'fibo'\n\nIf you intend to use a function frequently, you can assign it to a local name:\n\n```python\n>>> fib = fibo.fib\n>>> fib(500)\n1 1 2 3 5 8 13 21 34 55 89 144 233 377\n\n* * *\n\n## The from … import Statement\n\nPython's `from` statement allows you to import a specific part from a module into the current namespace. The syntax is:\n\n```python\nfrom modname import name1[, name2[, ... nameN]]\n\nFor example, to import the `fib` function from the `fibo` module, use the following statement:\n\n```python\n>>> from fibo import fib, fib2\n>>> fib(500)\n1 1 2 3 5 8 13 21 34 55 89 144 233 377\n\nThis statement does not import the entire `fibo` module into the current namespace; it only imports the `fib` function from `fibo`.\n\n### Aliasing Modules\n\nUse the `as` keyword to create aliases for modules or functions:\n\n```python\nimport numpy as np # Alias the numpy module as np\nfrom math import sqrt as square_root # Alias the sqrt function as square_root\n\n* * *\n\n## The from … import * Statement\n\nIt is also possible to import all content from a module into the current namespace using the following statement:\n\n```python\nfrom modname import *\n\nThis provides a simple way to import all items from a module.\n\nThis is not recommended as it can easily cause naming conflicts.\n\n* * *\n\n## In-Depth into Modules\n\nBesides function definitions, modules can also include executable code. This code is generally used to initialize the module. It is only executed the first time the module is imported.\n\nEach module has its own independent symbol table, which is used as the global symbol table for all functions within the module.\n\nTherefore, the module author can safely use these global variables within the module without worrying about interfering with other users' global variables.\n\nOn the other hand, when you are sure of what you are doing, you can also access functions within the module using the notation `modname.itemname`.\n\nModules can import other modules. It is a convention (not a requirement) to use `import` at the beginning of a module (or script, or elsewhere) to import a module. The name of the imported module is placed in the symbol table of the current module.\n\nAnother way to import is to use `import` to directly import names (functions, variables) from a module into the current module. For example:\n\n```python\n>>> from fibo import fib, fib2\n>>> fib(500)\n1 1 2 3 5 8 13 21 34 55 89 144 233 377\n\nThis import method does not place the name of the imported module in the current symbol table (so in this example, the name `fibo` is not defined).\n\nThere is yet another method to import all (function, variable) names from a module into the current module's symbol table at once:\n\n```python\n>>> from fibo import *\n>>> fib(500)\n1 1 2 3 5 8 13 21 34 55 89 144 233 377\n\nThis imports all names, except those starting with a single underscore (`_`). In most cases, Python programmers do not use this method because names imported from other sources may overwrite existing definitions.\n\n* * *\n\n## The __name__ Attribute\n\nWhen a module is imported by another program for the first time, its main program will run.\n\nIf we want a certain block of code in a module not to execute when the module is imported, we can use the `__name__` attribute to make that block execute only when the module is run directly.\n\n```python\n#!/usr/bin/python3\n# Filename: using_name.py\n\nif __name__ == '__main__':\n print('Program is running by itself')\nelse:\n print('I am from another module')\n\nThe output is:\n\n$ python using_name.py\nProgram is running by itself\n$ python\n>>> import using_name\nI am from another module\n>>>\n\n**Note: Every module has a `__name__` attribute.**\n\n* If the module is run directly, the value of `__name__` is `'__main__'`.\n* If the module is imported, the value of `__name__` is the module's name.\n\nNote: **__name__** and **__main__** have double underscores underneath, like this: `_ _`.\n\n* * *\n\n## The dir() Function\n\nThe built-in function `dir()` can find all names defined within a module. It returns a list of strings:\n\n```python\n>>> import fibo, sys\n>>> dir(fibo)\n['__name__', 'fib', 'fib2']\n>>> dir(sys)\n['__displayhook__', '__doc__', '__excepthook__', '__loader__', '__name__', '__package__', '__stderr__', '__stdin__', '__stdout__', '_clear_type_cache', '_current_frames', '_debugmallocstats', '_getframe', '_home', '_mercurial', '_xoptions', 'abiflags', 'api_version', 'argv', 'base_exec_prefix', 'base_prefix', 'builtin_module_names', 'byteorder', 'call_tracing', 'callstats', 'copyright', 'displayhook', 'dont_write_bytecode', 'exc_info', 'excepthook', 'exec_prefix', 'executable', 'exit', 'flags', 'float_info', 'float_repr_style', 'getcheckinterval', 'getdefaultencoding', 'getdlopenflags', 'getfilesystemencoding', 'getobjects', 'getprofile', 'getrecursionlimit', 'getrefcount', 'getsizeof', 'getswitchinterval', 'gettotalrefcount', 'gettrace', 'hash_info', 'hexversion', 'implementation', 'int_info', 'intern', 'maxsize', 'maxunicode', 'meta_path', 'modules', 'path', 'path_hooks', 'path_importer_cache', 'platform', 'prefix', 'ps1', 'setcheckinterval', 'setdlopenflags', 'setprofile', 'setrecursionlimit', 'setswitchinterval', 'settrace', 'stderr', 'stdin', 'stdout', 'thread_info', 'version', 'version_info', 'warnoptions']\n\nIf no arguments are given, the `dir()` function lists all names currently defined:\n\n```python\n>>> a = [1, 2, 3, 4, 5]\n>>> import fibo\n>>> fib = fibo.fib\n>>> dir() # Get a list of attributes defined in the current module\n['__builtins__', '__name__', 'a', 'fib', 'fibo', 'sys']\n>>> a = 5 # Create a new variable 'a'\n>>> dir()\n['__builtins__', '__doc__', '__name__', 'a', 'sys']\n>>> del a # Delete variable name 'a'\n>>> dir()\n['__builtins__', '__doc__', '__name__', 'sys']\n>>>\n\n* * *\n\n## Standard Modules\n\nPython comes with some standard module libraries, which are described in the Python Library Reference (the "Library Reference" mentioned later).\n\n| Module Name | Functional Description |\n| --- | --- |\n| `math` | Mathematical operations (e.g., square root, trigonometric functions) |\n| `os` | Operating system-related functions (e.g., file and directory operations) |\n| `sys` | System-related parameters and functions |\n| `random` | Generate random numbers |\n| `datetime` | Handle dates and times |\n| `json` | Process JSON data |\n| `re` | Regular expression operations |\n| `collections` | Provide additional data structures (e.g., defaultdict, deque) |\n| `itertools` | Provide iterator tools |\n| `functools` | Higher-order function tools (e.g., reduce, lru_cache) |\n\nSome modules are built directly into the interpreter. Although these are not built-in language features, they are highly efficient and can even handle system-level calls.\n\nThese components are configured differently based on the operating system. For example, the `winreg` module is only available on Windows systems.\n\nNote that there is a special module `sys`, which is built into every Python interpreter. The variables `sys.ps1` and `sys.ps2` define the strings for the primary and secondary prompts:\n\n```python\n>>> import sys\n>>> sys.ps1\n'>>> '\n>>> sys.ps2\n'... '\n>>> sys.ps1 = 'C> '\nC> print('!')\n!\nC>\n\n* * *\n\n## Packages\n\nPackages are a way to manage Python module namespaces using "dotted module names."\n\nFor example, a module named `A.B` indicates a submodule `B` within a package `A`.\n\nJust as you don't have to worry about global variables in different modules interfering with each other when using modules, using dotted module names means you don't have to worry about module name conflicts between different libraries.\n\nThis way, different authors can provide a `NumPy` module or a Python graphics library.\n\nSuppose you want to design a set of modules (or a "package") for uniformly processing sound files and data.\n\nThere are many different audio file formats (basically distinguished by extension, e.g., `.wav`, `.aiff`, `.au`), so you need a growing set of modules to convert between different formats.\n\nAdditionally, there are many different operations for this audio data (such as mixing, adding echo, equalizer functions, creating artificial stereo effects), so you also need an endless set of modules to handle these operations.\n\nHere is a possible package structure (in a hierarchical file system):\n\nsound/ Top-level package\n __init__.py Initialize the sound package\n formats/ Subpackage for file format conversions\n __init__.py\n wavread.py\n wavwrite.py\n aiffread.py\n aiffwrite.py\n auread.py\n auwrite.py\n ...\n effects/ Subpackage for sound effects\n __init__.py\n echo.py\n surround.py\n reverse.py\n ...\n filters/ Subpackage for filters\n __init__.py\n equalizer.py\n vocoder.py\n karaoke.py\n ...\n\nWhen importing a package, Python searches for subdirectories within the package based on directories in `sys.path`.\n\nA directory is only recognized as a package if it contains a file named `__init__.py`, mainly to prevent common names (like `string`) from accidentally affecting valid modules in the search path.\n\nIn the simplest case, an empty `__init__.py` file is sufficient. Of course, this file can also contain initialization code or assign values to the `__all__` variable (which will be introduced later).\n\nUsers can import specific modules from a package each time, for example:\n\n```python\nimport sound.effects.echo\n\nThis will import the submodule: `sound.effects.echo`. It must be accessed using its full name:\n\n```python\nsound.effects.echo.echofilter(input, output, delay=0.7, atten=4)\n\nAnother way to import a submodule is:\n\n```python\nfrom sound.effects import echo\n\nThis also imports the submodule: `echo`, and it doesn't require the long prefix, so it can be used like this:\n\n```python\necho.echofilter(input, output, delay=0.7, atten=4)\n\nA variation is to directly import a function or variable:\n\n```python\nfrom sound.effects.echo import echofilter\n\nSimilarly, this method imports the submodule: `echo`, and its `echofilter()` function can be used directly:\n\n```python\nechofilter(input, output, delay=0.7, atten=4)\n\nNote that when using the form `from package import item`, the corresponding `item` can be a submodule (sub-package) within the package, or another name defined in the package, such as a function, class, or variable.\n\nThe `import` syntax will first treat `item` as a package-defined name. If not found, it will try to import it as a module. If still not found, it raises an **ImportError** exception.\n\nConversely, if using the form `import item.subitem.subsubitem`, except for the last item, all must be packages, and the last item can be a module or package, but not a class, function, or variable name.\n\n* * *\n\n## Importing * from a Package\n\nWhat happens if we use `from sound.effects import *`?\n\nPython will enter the file system, find all submodules in the package, and import them one by one.\n\nHowever, this method does not work very well on Windows because Windows is a case-insensitive system.\n\nOn Windows, we cannot determine whether a file named `ECHO.py` is imported as `echo`, `Echo`, or `ECHO`.\n\nTo solve this problem, we only need to provide an explicit package index.\n\nImport statements follow these rules: If the package definition file `__init__.py` contains a list variable named `__all__`, then when using `from package import *`, all names in this list are imported as package contents.\n\nAs a package author, don't forget to ensure `__all__` is updated after updating the package.\n\nThe following example in `sound/effects/__init__.py` contains the following code:\n\n```python\n__all__ = ["echo", "surround", "reverse"]\n\nThis means that when you use `from sound.effects import *`, you will only import these three submodules from the package.\n\nIf `__all__` is not defined, then using `from sound.effects import *` will not import any submodules from the `sound.effects` package. It only imports the package `sound.effects` and all content defined within it (possibly running initialization code defined in `__init__.py`).\n\nThis will import all names defined in `__init__.py`. And it will not break all explicitly imported modules before this statement. Look at this code:\n\n```python\nimport sound.effects.echo\nimport sound.effects.surround\nfrom sound.effects import *\n\nIn this example, before executing `from...import`, the `echo` and `surround` modules in the `sound.effects` package are already imported into the current namespace. (Of course, if `__all__` is defined, it's even better.)\n\nUsually, we do not advocate using the `*` method to import modules because it often reduces code readability. However, it does save a lot of typing, and some modules are designed to be imported only through specific methods.\n\nRemember, using `from Package import specific_submodule` is never wrong. In fact, this is the recommended method. Unless the submodule you want to import might conflict with submodules from other packages.\n\nIf in the structure, a package is a sub-package (for example, in this case, for the package `sound`), and you want to import a sibling package (a package at the same level), you must use an absolute import path. For example, if the module `sound.filters.vocoder` wants to use the `echo` module from the `sound.effects` package, you must write `from sound.effects import echo`.\n\n```python\nfrom . import echo\nfrom .. import formats\nfrom ..filters import equalizer\n\nBoth implicit and explicit relative imports start from the current module. The main module's name is always `"__main__"`, and a Python application's main module should always use absolute path references.\n\nPackages also provide an additional attribute `__path__`. This is a list of directories, each containing an `__init__.py` that serves this package. You must define it before other `__init__.py` files are executed. This variable can be modified to affect the modules and sub-packages contained in the package.\n\nThis feature is not commonly used and is generally used to extend modules within a package.