YouTip

Time series analysis is an important part of data analysis. Pandas provides rich functionality for processing and analyzing time series data, including resampling, rolling calculations, moving averages, and more.\n\n* * *\n\n## Basic Time Series Operations\n\n### Creating Time Series\n\n## Example\n\nimport pandas as pd\n\nimport numpy as np\n\n# Create time series data\n\n np.random.seed(42)\n\n dates = pd.date_range("2024-01-01", periods=100, freq="D")\n\nts = pd.Series(\n\n np.random.randn(100).cumsum() + 100,\n\n index=dates\n\n)\n\nprint("Time series data (first 10 rows):")\n\nprint(ts.head(10))\n\nprint()\n\n# View information\n\nprint(f"Index type: {type(ts.index)}")\n\nprint(f"Start time: {ts.index.min()}")\n\nprint(f"End time: {ts.index.max()}")\n\nprint(f"Time span: {ts.index.max() - ts.index.min()}")\n\n### Setting Date as Index\n\n## Example\n\nimport pandas as pd\n\nimport numpy as np\n\n# Create DataFrame and set date index\n\n df = pd.DataFrame({\n\n"Date": pd.date_range("2024-01-01", periods=30, freq="D"),\n\n"Sales Revenue": np.random.randint(100,500,30),\n\n"Visitor count": np.random.randint(50,200,30)\n\n})\n\nprint("Before setting:")\n\nprint(df.head())\n\nprint()\n\n# Set date as index\n\n df = df.set_index("Date")\n\nprint("After setting:")\n\nprint(df.head())\n\nprint()\n\n# Use loc to query by date\n\nprint("Query 2024-01-05 to 2024-01-10:")\n\nprint(df.loc["2024-01-05":"2024-01-10"])\n\n* * *\n\n## Resampling\n\nResampling is the process of converting time series data from one frequency to another, including upsampling (increasing data points) and downsampling (decreasing data points).\n\n### Downsampling\n\n## Example\n\nimport pandas as pd\n\nimport numpy as np\n\n# Create daily level data\n\n np.random.seed(42)\n\n dates = pd.date_range("2024-01-01", periods=90, freq="D")\n\n ts = pd.Series(np.random.randint(100,500,90), index=dates)\n\nprint("Daily level data (first 10 rows):")\n\nprint(ts.head(10))\n\nprint()\n\n# Resample by month (sum)\n\n monthly = ts.resample("M").sum()\n\nprint("Monthly sum:")\n\nprint(monthly)\n\nprint()\n\n# Resample by month (mean)\n\n monthly_mean = ts.resample("M").mean()\n\nprint("Monthly mean:")\n\nprint(monthly_mean)\n\n### Upsampling\n\n## Example\n\nimport pandas as pd\n\n# Create low frequency data\n\n dates = pd.date_range("2024-01-01", periods=3, freq="M")\n\n ts = pd.Series([100,200,150], index=dates)\n\nprint("Monthly level data:")\n\nprint(ts)\n\nprint()\n\n# Upsample to daily level (requires fill method)\n\n ts_daily = ts.resample("D").ffill()\n\nprint("Upsampled to daily level (first 10 rows):")\n\nprint(ts_daily.head(10))\n\n### Common Resampling Methods\n\n| Method | Description |\n| --- | --- |\n| `sum()` | Sum |\n| `mean()` | Mean |\n| `max() / min()` | Maximum/Minimum |\n| `first() / last()` | First/Last value |\n| `count()` | Non-null count |\n| `ohlc()` | Open, High, Low, Close |\n\n* * *\n\n## Rolling Calculations\n\nRolling calculations perform sliding window computations on time series data, commonly used for calculating moving averages, moving standard deviations, etc.\n\n### Moving Average\n\n## Example\n\nimport pandas as pd\n\nimport numpy as np\n\n# Create data\n\n np.random.seed(42)\n\n dates = pd.date_range("2024-01-01", periods=30, freq="D")\n\n ts = pd.Series(np.random.randint(100,200,30), index=dates)\n\n# Calculate 7-day moving average\n\n rolling_mean = ts.rolling(window=7).mean()\n\nprint("7-day moving average (first 10 rows):")\n\nprint(rolling_mean.head(10))\n\nprint()\n\n# Calculate 7-day moving standard deviation\n\n rolling_std = ts.rolling(window=7).std()\n\nprint("7-day moving standard deviation:")\n\nprint(rolling_std.head(10))\n\n### Rolling Apply Custom Functions\n\n## Example\n\nimport pandas as pd\n\nimport numpy as np\n\nts = pd.Series(range(1,11))\n\nprint("Original data:")\n\nprint(ts)\n\nprint()\n\n# Rolling sum\n\nprint("Rolling 3 sum:")\n\nprint(ts.rolling(3).sum())\n\nprint()\n\n# Rolling maximum\n\nprint("Rolling 3 maximum:")\n\nprint(ts.rolling(3).max())\n\nprint()\n\n# Use apply\n\nprint("Rolling 3 custom function (range):")\n\nprint(ts.rolling(3).apply(lambda x: x.max() - x.min()))\n\n* * *\n\n## Time Series Visualization\n\n## Example\n\nimport pandas as pd\n\nimport numpy as np\n\n# Create sample data\n\n np.random.seed(42)\n\n dates = pd.date_range("2024-01-01", periods=100, freq="D")\n\n ts = pd.Series(\n\n np.random.randn(100).cumsum() + 100,\n\n index=dates\n\n)\n\n# Calculate moving averages\n\n ma_7 = ts.rolling(7).mean()\n\n ma_30 = ts.rolling(30).mean()\n\n# Display data\n\nprint("Time series + Moving averages:")\n\nprint(f"Original data (first 5 rows): {ts.head().tolist()}")\n\nprint(f"7-day MA (first 5 rows): {ma_7.dropna().head().tolist()}")\n\nprint(f"30-day MA (last 5 rows): {ma_30.dropna().tail().tolist()}")\n\n* * *\n\n## Time Series Feature Extraction\n\n## Example\n\nimport pandas as pd\n\n# Create time series\n\n ts = pd.Series(\n\n[1,2,3,4,5,6,7,8,9,10],\n\n index=pd.date_range("2024-01-01", periods=10, freq="D")\n\n)\n\n# Difference\n\n diff = ts.diff()\n\nprint("First-order difference:")\n\nprint(diff)\n\nprint()\n\n# Percentage change\n\n pct = ts.pct_change()\n\nprint("Percentage change:")\n\nprint(pct)\n\nprint()\n\n# Shift\n\n shifted = ts.shift(1)\n\nprint("Shift back by 1:")\n\nprint(shifted)\n\n* * *\n\n## Practical Example: Stock Data Analysis\n\n## Example\n\nimport pandas as pd\n\nimport numpy as np\n\n# Simulate stock data\n\n np.random.seed(42)\n\n n_days =60\n\ndf = pd.DataFrame({\n\n"Date": pd.date_range("2024-01-01", periods=n_days, freq="D"),\n\n"Opening Price": 100 + np.random.randn(n_days).cumsum(),\n\n"Closing Price": 100 + np.random.randn(n_days).cumsum(),\n\n"Volume": np.random.randint(1000000,10000000, n_days)\n\n})\n\n df = df.set_index("Date")\n\n# Calculate daily returns\n\n df= df.pct_change()\n\n# Calculate volatility (7-day rolling standard deviation)\n\n df= df.rolling(7).std() * np.sqrt(252)# Annualized\n\n# Calculate moving averages\n\n df= df.rolling(5).mean()\n\n df= df.rolling(20).mean()\n\n# Generate trading signals (golden cross/death cross)\n\n df=0\n\n df.loc[df> df,"Signal"]=1\n\n df.loc[df< df,"Signal"]= -1\n\nprint("Stock data analysis results:")\n\nprint(df.tail(10))\n\n* * *\n\n## Common Issues\n\n**1. Discontinuous Dates**\n\nSome time series don't have data for every day (holidays, etc.), requiring `asfreq` or `reindex` for handling.\n\n**2. Timezone Issues**\n\nWhen handling cross-timezone data, use `tz_localize` and `tz_convert` for timezone setting and conversion.\n\n**3. Missing Value Impact**\n\nRolling calculations skip missing values by default, which may affect result continuity.\n\n> Time series analysis is a foundational skill in finance, meteorology, IoT, and other fields. Pandas provides a complete toolchain for processing such data.