How do you keep the quality of a 70-billion-parameter model while running up to 20 times as fast? Combine two techniques that address different bottlenecks. Knowledge distillation transfers knowledge from a large teacher to a compact student, cutting model size and cost. Medusa reduces latency at inference time by adding prediction heads that decode multiple tokens simultaneously, taking advantage of GPUs' native parallelism. Working with Booking.com, AWS researchers unified both in a single pipeline and applied it across three travel features. The final model matched the quality of the best open-source LLMs at less than one-third the cost, using natural language to filter results 15 times as fast for 99% of queries: https://amzn.to/3Wn7umC
Amazon Science
Research Services
Seattle, Washington 397,195 followers
The latest news and research from Amazon’s science community. #AmazonScience
About us
Amazon Science gives you insight into the company’s approach to customer-obsessed scientific innovation. Amazon fundamentally believes that scientific innovation is essential to being the most customer-centric company in the world. It’s the company’s ability to have an impact at scale that allows us to attract some of the brightest minds in artificial intelligence and related fields. Our scientists continue to publish, teach, and engage with the academic community, in addition to utilizing our working backwards method to enrich the way we live and work. Follow us on LinkedIn and visit our website to get a deep dive on innovation at Amazon, and explore the many ways you can engage with our scientific community. #AmazonScience
- Website
-
https://www.amazon.science
External link for Amazon Science
- Industry
- Research Services
- Company size
- 10,001+ employees
- Headquarters
- Seattle, Washington
- Specialties
- Artificial Intelligence, Machine Learning, Computer Vision, Cloud, Economics, Sustainability, AI, ML, Conversational AI, Natural Language Processing, NLP, Robotics, Security, Privacy, Information, Knowledge Management, Operations, Scientific Research, Search, Amazon, and Alexa
Updates
-
Last week, over a thousand Amazonians from the research, science, and builder communities came together for the annual Amazon Machine Learning Conference (AMLC). Running since 2013, this year's AMLC featured over 150 sessions, including internal oral presentations, tutorials, workshops, and keynotes. A few scenes from the week below.
-
-
-
-
-
+7
-
-
The deadline for the Amazon Research Awards Fall 2026 call for proposals has been extended to November 11. Successful applicants receive unrestricted funds, AWS promotional credits, and training resources, including AWS tutorials and hands-on sessions with Amazon scientists and engineers. Submit your proposal today: https://amzn.to/4koPmm9
-
Amazon's custom Trainium chip helps run the diffusion-model inference behind Prime 360's cinematic highlights. Prime 360 presents actual game footage through diffusion models, which transform noise into a polished frame through iterative denoising. Each iteration involves billions of mathematical operations: matrix multiplications, attention computations, and denoising steps that must keep pace with a live broadcast. Trainium's "tile in, tile out" architecture is built for exactly this, optimized for workflows whose inputs and outputs are large tensors. The diffusion model splits each frame into dozens of overlapping tiles, each of which passes through the model independently. The result is diffusion-model inference that reduces latency by more than 40%: https://amzn.to/4e1puZN
-
-
Hybrid models increase LLM efficiency by replacing selected attention layers with mechanisms like sliding-window attention and state space models, reducing the memory footprint while preserving quality. But until now, exploring hybrid designs required training every new architecture from scratch. AWS researchers introduce "priming," a method that leverages pretrained Transformer weights to reduce the pretraining cost of hybrid architectures by more than 99%. The technique identifies which attention layers can be faithfully compressed into fixed-size state space layers, transfers weights, and fine-tunes the full hybrid with less than 0.5% of the source model's original token budget. The controlled comparison reveals a clear expressiveness hierarchy among state space models: Gated KalmaNet outperforms Gated DeltaNet, which outperforms Mamba-2. At 32 billion parameters, the best hybrid matches its source Transformer on reasoning while decoding up to 2x as fast at 128K context: https://amzn.to/3ULUkz6
-
-
This week, we announced the availability of TorchNeuron, an open-source backend that translates PyTorch code into operations executable on AWS Trainium systems. PyTorch code that already runs on other accelerators will run on Trainium with minimal or no code changes. Runtime optimizations and a thorough evaluation process driven by agentic AI help ensure that models written in PyTorch will execute efficiently and reliably on Trainium systems. Learn how the team used AI agents and hundreds of thousands of open-source models to validate TorchNeuron: https://amzn.to/4AaVbIT
-
-
When a network fails, the bottleneck is not engineering expertise. It is cognitive overload. A new approach augments a network graph with agentic AI to produce a "digital twin" that can help isolate network failures. The system runs a three-stage cascade of graph algorithms: decomposition narrows thousands of candidate nodes to hundreds, community detection clusters those to tens, and topology-aware centrality ranking identifies the most likely root cause. The key shift is conditioning every centrality measure on the specific failure, not the full graph. Personalized PageRank seeds random walks from nodes that are issuing alarms. Degree centrality counts only edges to those nodes. Closeness measures distance to the failure cluster, not the geometric center: https://amzn.to/4s8P0lr
-
The biggest barrier to adopting new AI hardware is the months spent rewriting and revalidating code that already works. Announcing TorchNeuron, which lets PyTorch code run natively on AWS Trainium chips with a single configuration change. No rewrite, and the tools developers already use to test, debug, and scale their models all work on Trainium without modification. Startups and enterprises in robotics, video, world models, self-driving cars, speech, drug discovery, LLMs, and finance are already building on it. TorchNeuron will be open sourced at the PyTorch Conference on October 20: https://amzn.to/4icd1Ut
-
Building trust into computing grows more urgent as AI becomes more autonomous and systems become more complex. The Amazon Trustworthy Computing Symposium brought together Amazon researchers and academic leaders in Arlington for a day of work on provably correct and secure systems, trustworthy agentic AI, and privacy-preserving technology. Across every topic, the same principle held: rigor is required at every layer, from hardware foundations to the safeguards governing autonomous agents. Thank you to all the speakers and participants for a day of meaningful research exchange. Nickolai Zeldovich Ruzica Piskac Ioana Boureanu Carlson Lingming Zhang Amin Kharraz Yiling Lou Yuan Tian Joe Calandrino Baris Coskun Jim Grundy Boaz Barak
-
-
-
-
-
+3
-
