Discussing ChatGPT with the World Bank: Questions on research, knowledge management, bias and creativity
On May 3, 2023, I had the pleasure to talk to the World Bank Group's Community of Practice on Knowledge Management about the opportunities that ChatGPT and similar tools present to the KM space. The group was exceptionally astute and came up with a number of very important questions that I believe are critical to reflect on. In the following, I am showcasing a few of them, along with my own thoughts that I articulated in the exchange:
Question 1: I am interested to know if systems like ChatGPT can generate valuable new knowledge, i.e. combine lessons from a few past projects and suggest how should we approach the upcoming project?
Generative models like GPT-4 will always build on existing bodies of knowledge, and therefore are (at the moment at least) better suited for a look into the past than as a tool for foresight or predictive analysis. However, as you suggest, we can ask it to combine information, theories, frameworks and models that have never been combined before and GPT-4 (GPT 3.5 much less so) will come up with interesting hypotheses and conclusions (Warton School Professor Ethan Mollick came up with some great examples). Some of the results might be bland, but we might come across some actual unique insights as well. In most cases the insight could be achieved by a human if he/her made the effort to think through that combination of information. So its often a function of the limited time humans have to think up hypotheses and search for such insights in the data. Now with Large Language Models dramatically reducing that time, it is mostly a question of coming up with new sets of data and the right questions. My personal expectation is that in the future a large part of quantitative research will be automated, and AI will be generating new insights at a rapid pace. Whether it works for leveraging and aggregating qualitative lessons learned form past project work (also my personal dream since I started working in KM 18 years ago) is not entirely clear yet, and probably depends on the quality of those lessons (i.e. do those lessons go deeper than just the usual “we learned that we should include all stakeholders form the beginning?)
Question 2: Can systems like ChatGPT substitute search engines?
I would see it as a powerful complement (or augmentation) rather than a replacement. We can’t fully rely on Large Language Models (LLMs) to do search because they’re prone to hallucinations and that’s a difficult problem to solve (if it ever can be fully solved, since LLMs that don’t hallucinate might also not be creative anymore). At the same time, we are probably shifting away from the long list of search results that Google gives us. We likely end up somewhere between (and something which Microsoft is trying with its GPT-enhanced Bing Chat) in which a semantic search engine will find results the traditional way but an LLM will analyze and consolidate that into a response for us (along with reference that back up its answer).
Question 3: Could you speak to biases in ChatGPT (and AI in general) and how to catch/prevent these, please?
The content base of the web is predominantly Western, English-speaking and white. That’s a problem, even though it will somewhat change over time (e.g. the number of English speaking internet users and content creators from Southern Asia is rising rapidly). OpenAI and its competitors are working on preventing the worst excesses of these biases, and the systems will get better in this regard over time. But the truth is, some biases will always be there because humans are inherently biased and AI is trained on content created by humans. The best weapon against this is personal awareness that any content we come across (whether it be from humans or AI) is likely biased in some way and that we ourselves are biased, and to continuously work on ourselves and our organisational culture and practices to sharpen our instincts and re-train ourselves. Btw, this will also be our most important tool against the flood of misinformation we are about to experience due to automated AI-generated content at scale.
Question 4: What are the implications of KM practitioners work with Communities of Practice?
Many! For one, no one has to write consolidated replies anymore, GPT-powered systems can do that in the future on the fly right after every discussion. It will also be much easier for communities to create collaborative products, since we can crowdsource inputs on a topic from various community members in brief bullet points and let ChatGPT turn that into a blog post, paper or concept note. But I don’t think it will replace the actual interaction in comments threads and discussion threads and webinars. In fact, those discussions will be a vital knowledge base for AI-generated outputs within the organization. If anything, that makes the case for building and nurturing vibrant communities within organizations even stronger.
Question 5: How can Development Finance Institutions (DFI) leverage generative AI to stimulate economic growth in emerging markets?
I’m not an economist obviously, but from the top of my head, I could see governments using ChatGPT (or customized GPT-4 powered systems) for scenario planning based on economic data. They might want to introduce legislation that mandates data transparency and accessibility (since data will be a key resource in the AI age) while counterbalancing that with suitable privacy and cybersecurity laws. Generative AI for personalized healthcare will also be a huge opportunity for populations in regions without easy access to doctors, and the same goes for education where personalized tutoring via AI could be a blessing for students and teachers alike. This all of course depends on ubiquitous access to mobile phones and mobile web. One way to boost this might be via regulation and projects that foster implementation of leight-weight open source AI systems (the stable diffusion approach) as opposed to relying on a few gargantuan players in the tech space like Microsoft, Google and Apple, and their proprietary hardware and software. Personally, I am particularly concerned with proposed legislation (such as the EU AI Act) that seems to favor large tech companies in developed countries who have the means to pay for expensive licensing and testing their models as well the associated fines at the expense of the very vibrant AI open source community, which would be practically barred from AI development going forward. It's the wrong approach in my opinion.
Question 6: Your views on AI and creativity. Are we moving too fast? Should we stop and let human beings keep on using their brains and their creativity?
There are many valid concerns regarding AI, but I don’t think AI is a threat to creativity. Everyone who took the metaphorical surfboard and threw themselves into this wave has done so by fully investing their creativity and humanness into it and as result amplifying their creativity. We are seeing right now a Cambrian explosion of new tools and apps, new research, discussions and think pieces, new forms of storytelling, pictoral and animated art as well as games. And a lot of that by people who didn’t have the means or craft to express their creative vision before. We see people who can’t code programming apps and games. People who can’t draw or paint creating graphic novels or design fashion or houses. People who can’t animate make videos and people who can’t sing or play an instrument write and produce new songs. People haven’t produced creative outputs in the past because it was a chore, but because they wanted to express themselves. I don’t see why that inherent drive of humans to express themselves and their ideas would go away with new tech (it didn’t happen with the printing press, photography, radio, TV or the web – despite similar forecasts by critics at the time). Rather the opposite: AI, like every tech before, will amplify our capabilities to express ourselves and open up new avenues and techniques to do so.
Do you want to explore these and other questions regarding Generative AI with your organization or your team? Then reach out at IntegralKM@gmail.com and we can tailor a session to your needs, supported with hands-on examples on how to use ChatGPT and similar tools in your everyday work!
Comments