YouTip

## PyTorch `torch.get_deterministic_debug_mode` Reference In deep learning, reproducibility is critical for debugging, auditing, and scientific research. However, many GPU operations (such as certain convolutions and scatter/gather operations) are non-deterministic by default to maximize performance. PyTorch provides a deterministic debug mode to help developers identify and restrict these non-deterministic behaviors. The `torch.get_deterministic_debug_mode` function is used to query the current configuration of this debug mode. --- ## Function Definition ```python torch.get_deterministic_debug_mode() -> int ``` ### Return Values This function returns an integer representing the current deterministic debug level: | Return Value | Mode | Description | | :--- | :--- | :--- | | `0` | **Disabled** | Default behavior. Non-deterministic operations are allowed to run normally without any warnings or errors. | | `1` | **Warn** | Non-deterministic operations will throw a warning (`UserWarning`), but the program will continue execution. | | `2` | **Error** | Non-deterministic operations will throw a runtime error (`RuntimeError`), immediately halting execution. | --- ## Code Examples ### 1. Basic Usage: Checking the Default Mode By default, PyTorch does not enforce strict determinism. You can check the default state using the following code: ```python import torch # Get the current deterministic debug mode mode = torch.get_deterministic_debug_mode() print("Current deterministic debug mode:", mode) # Output: Current deterministic debug mode: 0 ``` ### 2. Setting and Verifying Deterministic Debug Mode To change the debug mode, you use `torch.set_deterministic_debug_mode(mode)`. Below is a complete workflow showing how to set the mode and verify it using `torch.get_deterministic_debug_mode()`. ```python import torch # 1. Set the debug mode to "Warn" (1) torch.set_deterministic_debug_mode(1) print("Updated mode (Warn):", torch.get_deterministic_debug_mode()) # 2. Set the debug mode to "Error" (2) torch.set_deterministic_debug_mode(2) print("Updated mode (Error):", torch.get_deterministic_debug_mode()) # 3. Reset back to "Disabled" (0) torch.set_deterministic_debug_mode(0) print("Reset mode (Disabled):", torch.get_deterministic_debug_mode()) ``` --- ## Practical Considerations ### Why use Deterministic Debug Mode? * **Debugging Discrepancies:** If your model yields different loss values or evaluation metrics across identical runs, setting the mode to `1` or `2` helps you pinpoint exactly which PyTorch operators are causing the variance. * **Strict Reproducibility:** Setting the mode to `2` (Error) ensures that your codebase is 100% deterministic. If a developer introduces a non-deterministic operator, the CI/CD pipeline or local test suite will fail immediately. ### Performance Impact Enforcing determinism often forces PyTorch to use slower, single-pass algorithms instead of highly optimized, parallelized non-deterministic algorithms. It is recommended to use debug modes `1` and `2` during development and debugging, and revert to `0` in production environments where speed is prioritized. ### Related Functions * `torch.use_deterministic_algorithms(bool)`: A related configuration that forces PyTorch operations to use deterministic algorithms when available. * `torch.set_deterministic_debug_mode(mode)`: The setter function corresponding to `torch.get_deterministic_debug_mode()`.