Principal Component Analysis (PCA) Part 1: Machine Learning Interview Prep 14
Principal Component Analysis (PCA) squeezes lots of information into fewer pieces, sort of like squishing a big sandwich into a smaller box. These pieces, called principal components, highlight the most important aspects of the data. By using PCA, we can understand complex data better and make decisions more easily, like unraveling a tangled ball of yarn to find the main threads.
Let’s check your basic knowledge of Principal Component Analysis (PCA). Here are 10 multiple-choice questions for you and there’s no time limit. Have fun!
Question 1: Which statements are correct about the Principal Component Analysis (PCA)? (Select two)
(A) PCA is a supervised learning algorithm
(B) PCA is an unsupervised learning algorithm
(C) PCA is a clustering algorithm
(D) PCA is a dimensionality-reduction technique
Question 2: What are the curses of dimensionality? (Select two)
(A) As the number of features increases, so does the number of samples, resulting in a complex model. This is known as the curse of dimensionality.
(B) Due to the high number of features, there is a potential that the ML model would overfit. As a result, the model performs badly on the test data because it becomes overly reliant on training data.
(C ) As the number of features decreases, so does the number of samples, resulting in a complex model. This is known as the curse of dimensionality.
(D) Due to the lower number of features, there is a potential that the ML model would overfit. As a result, the model performs badly on the test data because it becomes overly reliant on training data.
Question 3: Which statements are correct about the definition of PCA? (Select two)
(A) PCA is a dimensionality reduction approach that converts a small set of connected variables into a bigger set of unrelated variables known as principal components.
(B) PCA aims to keep extraneous features while retaining most of the dataset’s variability.
(C) PCA is a dimensionality reduction approach that converts a big set of connected variables into a smaller set of unrelated variables known as principal components.
(D) PCA aims to eliminate extraneous features while retaining most of the dataset’s variability.
Question 4: What’s the advantage of Dimensionality Reduction?
(A) Removes redundant features and noise. Less misleading data means model accuracy improves.
(B) Less data means less storage space required. Fewer dimensions mean less computing. Less data means that algorithms train faster.
(C) Helps visualize the data on 2D plots or 3D plots.
(D) All of the above.
Question 5: What are the drawbacks of Dimensionality Reduction? (Select two)
(A) Some information is lost, possibly degrading the performance of subsequent training algorithms.
(B) Transformed features are often hard to interpret.
(C) Helps visualize the data on 2D plots or 3D plots.
(D) Less data means less storage space required. Fewer dimensions mean less computing. Less data means that algorithms train faster.
Question 6: What are the assumptions of PCA?
(A) There needs to be a linear relationship between all variables because PCA is based on Pearson correlation coefficients. Adequate correlations are needed between the variables to be reduced to a smaller number of components.
(B) For PCA to produce a reliable result, large enough sample sizes are required.
(C ) There should be no significant outliers.
(D) All of the above.
Question 7: What are the Limitations of PCA?
(A) PCA doesn’t work well for non-linearly correlated data.
(B) PCA always finds orthogonal principal components. Sometimes, data demands non-orthogonal principal components to represent the data.
(C) PCA always considers the low variance components in the data as noise and recommends throwing away those components. But, sometimes those components play a major role in a supervised learning task.
(D) All of the above.
Question 8: What are the applications of PCA?
(A) Image compression, 2D or 3D visualization of multi-dimensional data, Pre-processing steps for other machine learning algorithms, such as clustering and classification
(B) Image classification, 2D or 3D visualization of multi-dimensional data, Pre-processing steps for other machine learning algorithms, such as clustering and classification
(C) Image compression, 2D or 3D visualization of multi-dimensional data, and clustering
(D) Image classification, clustering, and pre-processing steps for other machine learning algorithms, such as clustering and classification
Question 9: Which statement is one of the limitations of PCA?
(A) Removes redundant features and noise. Less misleading data means model accuracy improves.
(B) Less data means less storage space required. Fewer dimensions mean less computing. Less data means that algorithms train faster.
(C) Helps us visualize the data on 2D plots or 3D plots.
(D) PCA only permits batch processing, which implies that all data must fit in the main memory.
Question 10: Which statement is incorrect about PCA?
(A) PCA requires standardization because we must assign equal weights to all variables; otherwise, we may receive misleading recommendations.
(B) Rotation is a crucial PCA phase that entails changing the coordinate system for the primary components. If we do not rotate the components, the effect of PCA will be diminished.
(C) PCA is a statistical technique for reducing the dimensionality of a dataset. This is accomplished by linearly transforming the data into a new coordinate system where (most of) the variation in the data can be described with fewer dimensions than the initial data.
(D) PCA reduces the dimensionality of a dataset by non-linearly transforming the data into a new coordinate system where (most of) the variation in the data can be described with fewer dimensions than the initial data.
PCA Algorithm:
Step 1. Standardize the data to ensure that all variables have a mean of 0 and a standard deviation of 1.
Step 2. Calculate the covariance matrix of the standardized data. This matrix shows how each variable is related to every other variable in the dataset.
Step 3. Calculate the eigenvectors and eigenvalues of the covariance matrix. The eigenvectors represent the directions in which the data varies the most, while the eigenvalues represent the amount of variation along each eigenvector.
Step 4. Choose the principal components which are the eigenvectors with the highest eigenvalues. These components represent the directions in which the data varies the most.
Step 5. Transform the original data into the lower-dimensional space defined by the principal components.
The solutions will be published in Principal Component Analysis (PCA) Part 2: Machine Learning Interview Prep 15.
Happy learning. If you like the questions and enjoy taking the test, please subscribe to my email list for the latest ML questions, follow my Medium profile, and leave a clap for me. Feel free to discuss your thoughts on these questions in the comment section. Don’t forget to share the quiz link with your friends or LinkedIn connections. If you want to connect with me on LinkedIn: my LinkedIn profile.
The solution of Confusion Matrix: Machine Learning Interview Prep 13 — 1(B), 2(A), 3(C), 4(D), 5(A), 6(A, B), 7(A, B), 8(D), 9(D), 10(C).
References:
[1] StatQuest: Principal Component Analysis (PCA), Step-by-Step
[2] StatQuest: PCA main ideas in only 5 minutes!!!
[3] StatQuest: PCA — Practical Tips
[4] Principal Component Analysis (PCA) clearly explained
[5] StatQuest: PCA in Python
[6] Principal Component Analysis Interview Questions
[7] 17 Principal Component Analysis (PCA) Interview Questions (ANSWERED)
[8] What is Principal Component Analysis?
[9] Importance of rotation in PCA
