Mozilla Developer Network (MDN) has released AI Help, a beta tool aiming to streamline web developers' interactions with the platform and provide problem-solving assistance. Only users with an MDN Plus account can access AI Help for now. So far, the community reaction has not been overly positive, and a number of GitHub issues have been reported.
AI Help leverages GPT-3.5 of OpenAI's generative AI and MDN's latest content to generate responses. It offers a chat interface that allows developers to ask questions directly on MDN and receive answers with related articles from MDN for contextual help. The tool taps into MDN's extensive database, providing quick access to coding documentation, best practices and information.
A section in the launch blog post, titled "Under the Hood", explains that AI Help leverages two technologies that work well together: embeddings for similarity search and adding generative AI. Embeddings are numerical representations of words and sentences that capture meaning and relationships, where similar words have closely situated vectors. When users ask questions, a new embedding is generated for the query, and a similarity search is conducted to identify the relevant content within the MDN (Mozilla Developer Network). Once the question and content are identified, generative AI is employed to extract the answer from the content.
The AI Help chat interface includes a simplified feedback reporting system to encourage a collaborative and user-driven approach. Users can find a "Report an Issue with this Answer" link within the chat interface that directs them to create an issue in MDN's GitHub issue tracker. This process allows users to share concerns or inaccuracies, contributing to continuous improvement and refinement of the AI Help feature.
MDN had previously released AI Explain, another AI-powered tool designed to help readers understand and explain code blocks. However, following feedback from the community pointing out instances of incorrect answers, MDN has decided to pause AI Explain temporarily.
The introduction of AI Help and AI Explain has gained considerable attention within the developer community. A user named eveee, who opened a GitHub issue on the Yari repository (the platform code behind MDN Web Docs) highlighted a key concern, stating:
"MDN's new "ai explain" button on code blocks generates human-like text that may be correct by happenstance or may contain convincing falsehoods. This is a strange decision for a technical reference."
Another user called catleeball emphasized that instead of using AI Help, relying on domain experts to write the documentation is crucial:
"I strongly feel that the AI help feature is likely to cause much more damage than it would possibly help. I think the best path forward is to offer only documentation written by humans, ideally reviewed by people who have domain expertise."
In response to the feedback, MDN acknowledges the input and believes that LLM technology is immature. Hence, the platform has decided to pause AI Explain while encouraging AI Help users to provide feedback using the GitHub link provided in each response.
Other products within this space include GitHub Copilot, Tabnine, and Replit’s Ghostwriter.