LLMの本番運用は高コストであり、その多くは冗長な計算に起因します。この短期コースでは、オープンソースの推論フレームワークSGLangを使用して、既に行われた計算をキャッシュし将来のリクエストで再利用することで、無駄を排除する方法を学びます。共有コンテキストが多い場合、速度向上は急速に累積します。
A compromised version of the LiteLLM Python package (version 1.82.8) was briefly available on PyPI, capable of exfiltrating sensitive credentials like SSH keys and cloud secrets. The malicious package affected any project that depended on LiteLLM, though it was only available for about an hour before discovery.
A supply chain attack has compromised the popular npm axios HTTP client library with 300 million weekly downloads. Malicious versions install a remote access trojan, though some users may have avoided infection through version pinning or older installations. Security experts warn this is a live compromise affecting one of npm's most depended-on packages.
A series of supply chain attacks has affected npm and PyPI repositories within two weeks. The use of large language models is exacerbating these security issues, and existing mitigation measures are insufficient to address the problem.
A new phishing-as-a-service called Starkiller uses disguised links to load real login pages from target brands. It acts as a relay between victims and legitimate sites, forwarding usernames, passwords, and MFA codes to bypass security measures.
A security researcher discovered a vulnerability that allowed obtaining full administrator rights in a Replit clone. The vulnerability stemmed from running untrusted code in an insecure manner. This highlights the importance of proper security practices when executing external code.