{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":649014851,"defaultBranch":"main","name":"lit-llama","ownerLogin":"Motsepe-Jr","currentUserCanPush":false,"isFork":true,"isEmpty":false,"createdAt":"2023-06-03T14:02:52.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/68114783?v=4","public":true,"private":false,"isOrgOwned":false},"refInfo":{"name":"","listCacheKey":"v0:1685801231.895272","currentOid":""},"activityList":{"items":[{"before":"c2dff33b92601ae1df662fa5f8020a54c5ebe7e3","after":"a492ad0a841ba1625a77fc6cd425cda3667dac91","ref":"refs/heads/sparsegpt","pushedAt":"2023-06-08T19:34:27.543Z","pushType":"push","commitsCount":1,"pusher":{"login":"Motsepe-Jr","name":"challenger","path":"/Motsepe-Jr","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/68114783?s=80&v=4"},"commit":{"message":"sparseGPT\n\nThis is the sparseGPT code based on IST-DASLab project.\n\nI followed the same coding principles as used in the lit-llama gptq code.\n\nI created a file called sparsification which is the algorithm for SparseGPT and a folder called sparsify/sparsegpt.py to run the algorithm on the model in the checkpoint_path.\n\nThis is my first contribution to the project, If I missed some household admin I apologize in advance.\n\nAssuming you have a  model under checkpoints/open-llama/7B\n\nyou can run this command:\npython sparsify/sparsegpt.py --checkpoint_path checkpoints/lit-llama/7B/lit-llama.pth\n\nKey Notes:\n\n0. I used half n_samples (128-->64) due to memory requirement\n\n1. The SparseGPT paper was evaluated on models trained not using the chinchilla scaling law (Therefore my hypothesis is that some of the weights of those model were not useful, hence they were able to prune 50%). With Llama I only used 0.1 target sparsity.)\n\n2. The source code of SparseGPT consist of a quantization algorithm similar to GPTQ, however, I removed this code because we already have GPTQ in the lit-llama source code. If you would like me to include it, it is also okay I can include GPTQ under sparseGPT code.\n\nBefore you commit, please also test from your side, and let me know if you want me to solve any bug or integrate a specific feature\n\nThanks","shortMessageHtmlLink":"sparseGPT"}},{"before":null,"after":"c2dff33b92601ae1df662fa5f8020a54c5ebe7e3","ref":"refs/heads/sparsegpt","pushedAt":"2023-06-03T14:07:11.895Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"Motsepe-Jr","name":"challenger","path":"/Motsepe-Jr","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/68114783?s=80&v=4"},"commit":{"message":"Sparse GPT Algorithm\n\nThis is the sparseGPT code based on IST-DASLab project.\n\nI followed the same coding principles as used in the lit-llama gptq code.\n\nI created a file called sparsification  which is the algorithm for SparseGPT and a folder called sparsify/sparsegpt.py to run the algorithm on the the model in the checkpoint_path.\n\nThis is my first contribution to the project, If I missed some household admin I apologise in advance.\n\nKey Notes:\n\n    1. The source code of SparseGPT consist of the quantization algorithm similar to GPTQ, however I removed this code because we already have GPTQ in the lit-llama source code.\n\n2. I'm still in the waiting list for the Llama weights 7B.","shortMessageHtmlLink":"Sparse GPT Algorithm"}}],"hasNextPage":false,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAADPbFegAA","startCursor":null,"endCursor":null}},"title":"Activity · Motsepe-Jr/lit-llama"}