Attention Head Pruning for Model Compression - Coding problem | OneNoughtOne
Loading content...
Machine Learning Patterns
0/347 completed
Linear algebra
0/1
0/30
0/1
Vectors and Vector Spaces: The Language of Machine Learning
5
Coordinate Transformation Between Bases
Coordinate Transformation Between Bases
Closed-Form Linear Model Fitting
Column Compressed Sparse Matrix Format
Component-wise Vector Addition
Linear Kernel Computation (Dot Product Similarity)
Linear Transformation of a Vector
Square Matrix Determinant and Trace Evaluator
Matrix Dimension Restructuring
Computing the Multiplicative Inverse of a 2×2 Matrix
Matrix Row-Column Interchange
Scaling a Matrix by a Constant Factor
Polynomial Kernel Similarity Computation
Hyperplane Separation Width for Linear Classifiers
Three-Dimensional Vector Cross Product
Vector Inner Product Computation
Vector Projection onto a Line
Construct Diagonal Matrix from Vector
Column Space Basis Extraction
Doubly Stochastic Stream Mixing Forward Pass
Gauss-Seidel Linear System Solver
Gaussian Kernel Similarity Matrix
Hyperdimensional Row Encoding Using Bind-Bundle Operations
Low-Rank Adaptation Forward Computation
Pairwise Matrix Product Computation
Surprise-Driven Neural Memory Update with Momentum
Two-Dimensional Orthonormal Basis Generator
Row Reduced Canonical Form
Similarity Matrix Transformation
Spatial Translation Transformation in 2D
Spectral Analysis of a Square Matrix
Probability and statistics
0/1
0/31
0/1
Calculus and optimization
0/1
0/65
0/1
Feature preparation
0/1
0/13
0/1
Classical machine learning
0/1
0/16
0/1
Neural networks and efficient training
0/1
0/22
0/1
Convolution and computer vision
0/1
0/22
0/1
Sequence models
0/1
0/3
0/1
Attention and transformers
0/1
0/16
0/1
Attention Mechanism
5
Gated Linear Unit with Swish Activation
Gated Linear Unit with Swish Activation
Attention Head Pruning for Model Compression
Attention Score Clipping Stabilizer
Causal Attention Mechanism
Two-Layer Feed-Forward Network with Skip Connection and Dropout
Gated Attention Mechanism
Localized Sliding Attention Mechanism
Scaled Dot-Product Attention
Self-Attention Weighted Aggregation
Sequence-Wise Feature Normalization with Affine Transform
Sparse Mixture-of-Experts: Dynamic Expert Selection and Aggregation
Autoregressive Text Synthesis Engine
Memory-Efficient Tiled Attention Forward Pass
Parallel Attention Heads Mechanism
Sparse Mixture of Experts Forward Pass
Transformer Position Embedding Generator
Generative models
0/1
0/5
0/1
Reinforcement learning
0/1
0/27
0/1
ML serving and evaluation
0/1
0/57
0/1
Text and information retrieval
0/1
0/5
0/1
Financial time series
0/1
0/4
0/1
Additional numerical and ML techniques
0/1
0/1
0/1
Course overview & roadmap
Toggle Sidebar
101
OneNoughtOne
0/347
Share