TR2026-093

EinSort: Sorting is All We Need for Tensorizing LLM

    •  Koike-Akino, T., Liu, J., Wang, Y., "EinSort: Sorting is All We Need for Tensorizing LLM", International Conference on Machine Learning (ICML) Workshop, July 2026.
      BibTeX TR2026-093 PDF
      • @inproceedings{Koike-Akino2026jul,
      • author = {Koike-Akino, Toshiaki and Liu, Jing and Wang, Ye},
      • title = {{EinSort: Sorting is All We Need for Tensorizing LLM}},
      • booktitle = {International Conference on Machine Learning (ICML) Workshop},
      • year = 2026,
      • month = jul,
      • url = {https://www.merl.com/publications/TR2026-093}
      • }
  • MERL Contacts:
  • Research Areas:

    Artificial Intelligence, Machine Learning

Abstract:

Tensor networks provide efficient representations for compressing large neural networks. By care- fully designing shapes and topologies, they can significantly reduce memory and computational costs. However, identifying implicit low-rank structures in large foundation models remains challenging due to their enormous scale and unstructured weight distributions. We propose an adaptive tensorization method that discovers inherent low-rank structure in a target tensor by index ordering. Experiments on weight and KV- cache compression demonstrate improved reconstruction quality compared to baselines.

 

  • Related Publication

  •  Koike-Akino, T., Liu, J., Wang, Y., "EinSort: Sorting is All We Need for Tensorizing LLM", arXiv, June 2026.
    BibTeX arXiv
    • @article{Koike-Akino2026jun,
    • author = {Koike-Akino, Toshiaki and Liu, Jing and Wang, Ye},
    • title = {{EinSort: Sorting is All We Need for Tensorizing LLM}},
    • journal = {arXiv},
    • year = 2026,
    • month = jun,
    • url = {https://arxiv.org/abs/2606.08565}
    • }