InfoQ Homepage Local Inference Content on InfoQ
News
RSS Feed-
FreeToken Unlocks Frontier MoE Inference on Consumer Hardware via Dynamic Co-Execution
Researchers from UC Berkeley and MIT have developed FreeToken, an open-source inference engine that enhances the utility of Mixture-of-Experts models on consumer hardware. By implementing a dynamic scheduling policy and optimising weight management, FreeToken improves decoding speeds and execution efficiency in edge AI applications, fostering self-hosted reasoning systems.
-
Meta Open-Sources Muse Glimmer: a 30B Local Agentic Model Optimised for On-Device Execution
Meta AI Research has introduced Muse Glimmer, a 30-billion-parameter open-weight model under the Apache 2.0 license, designed for local workflows. It enables autonomous agents and complex task execution on consumer GPUs without relying on cloud APIs. The model employs a multi-stage training approach for efficient performance and supports multimodal inputs, enhancing coding and automation tasks.