As the world of software development embraces the potential of generative AI tools like Amazon Web Services’ CodeWhisperer and GitHub’s Copilot, it becomes increasingly crucial for organizations to establish well-defined generative AI policies. These policies are essential not only to harness the productivity benefits but also to mitigate the potential risks associated with these tools.
Productivity boost with generative AI
Generative AI tools have shown immense promise in boosting developer productivity. However, the adoption of these tools without proper guidelines can have unintended consequences. Recent studies, such as one conducted by Purdue University researchers in August, have highlighted inaccuracies in generative AI models. Despite this, more than 80% of Fortune-500 companies are utilizing these tools, raising questions about code quality and reliability.
A notable example is Samsung, which faced an inadvertent leak of sensitive internal source code into a generative AI tool like ChatGPT. This incident led to a swift and sweeping ban on the use of generative AI assistants within the company. While such a reaction might seem reasonable in the short term, it lacks a long-term vision for harnessing the potential of generative AI.
To fully leverage the productivity potential of generative AI tools while avoiding public relations pitfalls, organizations must establish and communicate clear generative AI policies for their engineering teams. In this edition of Tech Works, we explore how engineering leadership that adopted generative AI early can guide organizations in creating effective policies.
Consumer vs. enterprise generative AI tools
Numerous generative AI tools are available, including CodeWhisperer, Google’s Bard, Meta AI’s LLaMA, Copilot, and OpenAI’s ChatGPT. However, tools like ChatGPT and Copilot have gained significant traction within engineering teams. The choice of which generative AI tool to use depends on the intended use case.
Consumer-focused tools like ChatGPT are often used without considering their limitations, resulting in inaccurate responses. Users must explicitly instruct these tools to provide answers only when they are certain, given that public LLMs are trained to provide responses even if they lack accurate information.
Using consumer-facing tools for proprietary or critical tasks within an organization can pose privacy and security risks. Therefore, organizations are advised to steer their engineers away from such tools and towards more secure and enterprise-oriented options.
The effectiveness of generative AI tools can be significantly enhanced by training them within the context of an organization’s specific needs. Internal developer chatbots can be trained on internal strategies, processes, and coding standards, which leads to more accurate and context-aware responses.
Generative AI is particularly valuable for tasks like generating code snippets, creating documentation, importing libraries, generating wireframes, running quality and security scans, and summarizing code. While these outputs may not always be production-ready, they serve as valuable starting points for developers.
Generative AI also offers substantial benefits in the realm of documentation. Internal documentation is often outdated, hard to access, and decoupled from the software development workflow. Generative AI can help by automatically regenerating code snippets, maintaining documentation, and assisting in context-driven searches.
Generative AI on Jobs
Generative AI is not intended to replace human developers but to augment their capabilities and streamline repetitive tasks. It enhances productivity by removing routine work, allowing developers to focus on higher-value activities.
Organizations need to recognize that generative AI is here to stay, and they should have well-defined generative AI policies in place. These policies should include training programs for engineers to effectively utilize generative AI tools, identify their strengths and weaknesses, and uphold ethical standards.
In a rapidly evolving landscape where generative AI tools are becoming commonplace, organizations that embrace these technologies with well-structured policies will likely enjoy increased productivity and remain at the forefront of software development.
