YouTipPython Pickle
## Python3.x Python Pickle Module
In Python development, we often need to save runtime objects or restore previous states after a program restarts, for example:
* Cache computation results to disk to avoid repeated calculations
* Save user configurations and program intermediate states
* Pass complex objects between different Python processes
The `pickle` module is the **official built-in solution** provided by Python to solve such problems.
Python's `pickle` module is a standard library module for serializing and deserializing Python objects.
Before starting to use Pickle, you need to understand two core concepts:
* Serialization (Pickling): Converting Python objects into byte sequences
* Deserialization (Unpickling): Converting byte sequences back into Python objects
The `pickle` module can save almost all Python objects (such as lists, dictionaries, class instances, etc.) to files, or transmit them over networks, and then reload them when needed.
### Why Use the Pickle Module?
1. **Data Persistence**: Save Python objects to files so that the data can still be accessed after the program closes.
2. **Data Transmission**: Transmit Python objects over networks, for example, passing data in distributed systems.
3. **Fast Storage and Loading**: The `pickle` module can efficiently handle complex data structures, suitable for scenarios requiring fast storage and loading.
### Typical Use Cases for Pickle
`pickle` is very suitable for the following scenarios:
1. Local data persistence
2. Program runtime state saving and recovery
3. Intermediate computation result caching
4. Python inter-process communication (IPC)
5. Saving machine learning models and feature data
Not suitable for scenarios:
* Cross-language data exchange
* Frontend-backend interface data transmission
* Deserialization of untrusted data sources
### What Objects Does Pickle Support?
#### 1. Supported Object Types
| Type | Supported |
| --- | --- |
| int / float / bool / str | Supported |
| list / tuple / dict / set | Supported |
| None | Supported |
| Custom class instances | Supported |
| Nested structures | Supported |
### Unsupported or Not Recommended Objects
* Open file objects
* Socket, database connections
* Operating system resources
* Objects that depend on runtime environment state
### Importing the Module
Using the Pickle module is very simple, just import it:
import pickle
* * *
### 1. Serializing Objects
Use the `pickle.dump()` method to serialize Python objects and save them to files.
## Example
import pickle
# Create a Python object
data ={
'name': 'Alice',
'age': 25,
'hobbies': ['reading','traveling']
}
# Serialize the object and save to file
with open('data.pkl','wb')as file:
pickle.dump(data,file)
* `'wb'` means open the file in binary write mode.
* `pickle.dump()` serializes the `data` object and writes it to the file.
### 2. Deserializing Objects
Use the `pickle.load()` method to load and deserialize Python objects from files.
## Example
import pickle
# Load and deserialize object from file
with open('data.pkl','rb')as file:
loaded_data =pickle.load(file)
print(loaded_data)
* `'rb'` means open the file in binary read mode.
* `pickle.load()` reads the byte stream from the file and deserializes it into a Python object.
### 3. Serializing to Byte String
If you don't want to save to a file, you can use pickle.dumps() to serialize objects to byte strings:
## Example
import pickle
data =[1,2,3,4,5]
# Serialize to byte string
byte_data =pickle.dumps(data)
print(byte_data)
# Output similar to: b'x80x04x95x0fx00x00x00...'
### 4. Deserializing from Byte String
Use pickle.loads() to deserialize objects from byte strings:
## Example
import pickle
byte_data = b'x 80x 04x 95x 0fx 00x 00x 00x 00x 00x 00x 00]x 94(Kx 01Kx 02Kx 03Kx 04Kx 05e.'
# Deserialize from byte string
original_data =pickle.loads(byte_data)
print(original_data)
# Output: [1, 2, 3, 4, 5]
### 5. Object Types That Can Be Serialized
Pickle can serialize most Python objects, including:
* Basic data types: integers, floats, strings, booleans, None
* Collection types: lists, tuples, dictionaries, sets
* Instances of custom classes
* Functions and classes (with some limitations)
## Example
import pickle
# Serialize different types of data
numbers =[1,2,3]
text ="Hello, Pickle"
dictionary ={'key': 'value'}
tuple_data =(1,2,3)
set_data ={1,2,3}
# Put all data into a list
all_data =[numbers, text, dictionary, tuple_data, set_data]
# Serialize
with open('mixed_data.pkl','wb')as f:
pickle.dump(all_data, f)
# Deserialize
with open('mixed_data.pkl','rb')as f:
loaded_data =pickle.load(f)
print(loaded_data)
### 6. Serializing Custom Classes
Pickle can handle instances of custom classes very well:
## Example
import pickle
class Student:
def __init__ (self, name, age, grade):
self.name= name
self.age= age
self.grade= grade
def __repr__ (self):
return f"Student(name={self.name}, age={self.age}, grade={self.grade})"
# Create instance
student = Student("Li Si",20,"Junior Year")
# Serialize
with open('student.pkl','wb')as f:
pickle.dump(student, f)
# Deserialize
with open('student.pkl','rb')as f:
loaded_student =pickle.load(f)
print(loaded_student)
# Output: Student(name=Li Si, age=20, grade=Junior Year)
### 7. Serializing Multiple Objects
You can call pickle.dump() multiple times to save multiple objects:
## Example
import pickle
data1 ={'item': 'apple','count': 5}
data2 =['banana','orange','grape']
data3 =42
# Save multiple objects
with open('multiple.pkl','wb')as f:
pickle.dump(data1, f)
pickle.dump(data2, f)
pickle.dump(data3, f)
# Read multiple objects (order must be consistent)
with open('multiple.pkl','rb')as f:
loaded_data1 =pickle.load(f)
loaded_data2 =pickle.load(f)
loaded_data3 =pickle.load(f)
print(loaded_data1)
print(loaded_data2)
print(loaded_data3)
### 8. Practical Application Examples
Example 1: Saving and Loading Machine Learning Model Configuration
## Example
import pickle
# Simulate machine learning model configuration
model_config ={
'model_type': 'RandomForest',
'n_estimators': 100,
'max_depth': 10,
'trained_date': '2024-01-13',
'accuracy': 0.95
}
# Save configuration
with open('model_config.pkl','wb')as f:
pickle.dump(model_config, f)
# Load configuration
with open('model_config.pkl','rb')as f:
config =pickle.load(f)
print(f"Model Type: {config['model_type']}")
print(f"Accuracy: {config['accuracy']}")
Example 2: Caching Computation Results
## Example
import pickle
import os
def expensive_computation(n):
"""Simulate time-consuming computation"""
result =sum(i ** 2 for i in range(n))
return result
def compute_with_cache(n, cache_file='cache.pkl'):
# Check if cache exists
if os.path.exists(cache_file):
with open(cache_file,'rb')as