YouTip

## Support Vector Machine Support Vector Machine (SVM) is a supervised learning algorithm, mainly used for classification and regression problems. The core idea of SVM is to find an optimal hyperplane that separates data of different categories. This hyperplane must not only correctly classify the data but also maximize the margin between the two categories. **Hyperplane**: * In two-dimensional space, a hyperplane is a straight line. * In three-dimensional space, a hyperplane is a plane. * In higher-dimensional space, a hyperplane is a hyperplane that divides the space. **Support Vectors**: * Support vectors are the sample points closest to the hyperplane. These support vectors are crucial for defining the hyperplane. * Support Vector Machine chooses the best hyperplane by maximizing the distance from support vectors to the hyperplane (i.e., maximizing the margin). **Maximum Margin**: * The goal of SVM is to maximize the classification margin, making the classification boundary as far away from both classes of data points as possible. This can effectively reduce the model's generalization error. **Kernel Trick**: * For non-linearly separable data, SVM uses kernel functions to map data to a higher-dimensional space, where the data may become linearly separable. * Common kernel functions include: linear kernel, polynomial kernel, Radial Basis Function (RBF) kernel, etc. ### SVM Classification Process 1. **Choose a hyperplane**: Find a hyperplane that can maximize the classification boundary. 2. **Train support vectors**: Through the Support Vector Machine algorithm, select the sample points closest to the hyperplane as support vectors. 3. **Find the optimal hyperplane by maximizing the margin**: Choose an optimal hyperplane that maximizes the margin. 4. **Use kernel functions to handle non-linear problems**: Map data to high-dimensional space through kernel functions to solve non-linearly separable problems. * * * ## Implementing SVM with Python Next, we will use the `scikit-learn` library in Python to implement a simple SVM classifier. ### 1. Install Required Libraries First, make sure you have installed the `scikit-learn` library. If not installed, you can install it using the following command: pip install scikit-learn ### 2. Import Libraries ## Example import numpy as np import matplotlib.pyplot as plt from sklearn import svm, datasets from sklearn.model_selection import train_test_split from sklearn.metrics import accuracy_score ### 3. Load Dataset We will use the Iris dataset that comes with `scikit-learn`. ## Example # Load Iris dataset iris = datasets.load_iris() X = iris.data[:, :2]# Take only the first two features y = iris.target ### 4. Split Training and Test Sets ## Example # Split dataset into training and test sets X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42) ### 5. Train SVM Model ## Example # Create SVM classifier clf = svm.SVC(kernel='linear')# Use linear kernel # Train model clf.fit(X_train, y_train) ### 6. Prediction and Evaluation ## Example # Make predictions on test set y_pred = clf.predict(X_test) # Calculate accuracy accuracy = accuracy_score(y_test, y_pred) print(f"Model accuracy: {accuracy:.2f}") ### 7. Visualize Results ## Example import numpy as np import matplotlib.pyplot as plt from sklearn import svm, datasets from sklearn.model_selection import train_test_split from sklearn.metrics import accuracy_score # Load Iris dataset iris = datasets.load_iris() X = iris.data[:, :2]# Take only the first two features y = iris.target # Split dataset into training and test sets X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42) # Create SVM classifier clf = svm.SVC(kernel='linear')# Use linear kernel # Train model clf.fit(X_train, y_train) # Make predictions on test set y_pred = clf.predict(X_test) # Calculate accuracy accuracy = accuracy_score(y_test, y_pred) print(f"Model accuracy: {accuracy:.2f}") # Plot decision boundary def plot_decision_boundary(X, y, model): h =.02# Grid step size x_min, x_max = X[:,0].min() - 1, X[:,0].max() + 1 y_min, y_max = X[:,1].min() - 1, X[:,1].max() + 1 xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h)) Z = model.predict(np.c_[xx.ravel(), yy.ravel()]) Z = Z.reshape(xx.shape) plt.contourf(xx, yy, Z, alpha=0.8) plt.scatter(X[:,0], X[:,1], c=y, edgecolors='k', marker='o') plt.xlabel('Sepal length') plt.ylabel('Sepal width') plt.title('SVM Decision Boundary') plt.show() plot_decision_boundary(X_train, y_train, clf) Execute the above code, output is: Model accuracy: 0.80 The image shows: !(#)