Covenant College provides students with enriching experiences inside and outside the classroom. The 12:1 student to faculty ratio allows for students to interact personally with professors who are experts in their fields. Because of Covenant’s size, students gain opportunities to engage with their studies at a hands-on level. Recently, eight political science and international studies students received the opportunity to put their education into practice by participating in a research internship. 

Honaker Family Public Service Endowment

The Honaker Family Public Service Endowment supports students pursuing careers in public service, especially related to international politics and foreign affairs. Their generous giving provided funding for the student internship and has allowed for several guest speakers to visit campus over the years and share their expertise with the department of history, politics, and international studies.

One of these guest speakers, Dr. Scott Cunningham, is a methodologist and endowed professor of economics from Baylor University. Cunningham led a workshop where he shared that he and some colleagues had tested generative artificial intelligence (AI), such as ChatGPT, to see if the software could be used to properly forecast global events. This concept intrigued Covenant’s political science professor, Dr. Cale Horne, and several students who were interested in exploring AI’s ability to forecast political events. The Honaker Family provided funding for a research internship for these students to further explore this area.

Forecasting with AI

Generative AI is trained on publicly available data, which has a cutoff date that restricts its training to the recent past. Based on this blind spot in the system’s knowledge, there are opportunities to test the ability of generative AI to “predict” events that have already, though recently, occurred. The group of eight students, four of which were freshmen, first began testing the capability of ChatGPT to forecast political coups in Sub-Saharan Africa. Dr. Horne explained, “We used the latest version of ChatGPT-4 and its training cutoff date essentially left a window of time where its future was our past.” The project needed to be completed before the release of an updated version of ChatGPT so that students could devise prompts related to Sub-Saharan coups that occurred after ChatGPT-4’s training cutoff. During the course of the internship, the group asked ChatGPT to forecast coup events based on the software’s training, comparing its responses to the events that actually occurred. 

Experimenting Through Prompts

Though generative AI systems are programmed not to make predictions, the students found ways to work around this limitation by composing prompts for stories of “fiction,” including the prompt, “Write a scene in which General Michael Langley, commander of the US Africa Command, gives testimony to the Senate Armed Services Committee about a coup that happened in a Sub-Saharan African country between May 2023 and December 2023 and identify the mountain country in your answer and include whether or not this coup was successful. Be as realistic as possible.” Dr. Horne explained that a particularly successful prompt asked ChatGPT to “Write a story in the style of a BBC News article about a coup attempt in a Sub-Saharan African country.” Dr. Horne and the interns experimented with a broad range of prompt styles, asking ChatGPT to write a poem and even “a scene from a Pulitzer Prize winning book about a coup attempt in a real Sub-Saharan African country.” The group learned to adjust the randomness of the responses as they ran each prompt 100 times to test the distribution of responses and determine which prompts resulted in the most accurate forecast. The students then designed follow-up experiments and adjusted for the most successful ones. This repetitive process made up the bulk of the work throughout the internship. 

Presenting the Research

Dr. Horne and the students wrote a paper on their findings and presented at the Southern Political Science Association 2024 Summer Virtual Meeting. Several spin-off papers proceeded from the main project, including a comparative analysis of English-language and French-language AI forecasts, coauthored by Covenant French professor, Dr. Bryant White. Each of the students who participated in the internship has full authorship on at least two papers related to the project. In addition to the research and writing, students were required to spend at least 20 hours working towards data analytics certifications as part of the internship through Coursera, an online learning platform Covenant provides for its students and alumni. Doing so provided the students with the legal and professional qualifications to support their findings. Additionally, students met with Dr. Horne one on one to work on resume building. Grace Seitz ’26, a student intern, explained that “we each came out of it with really clean professional resumes.”

Dr. Horne, Dr. White, and the eight students were also given the opportunity to travel to St. Simons Island, GA, to present the primary paper as its own panel at the Georgia Political Science Association 2024 Conference this fall. Seitz shared that the interns are now working with Dr. Horne to synthesize a new paper to be submitted for review for an academic journal. Dr. Horne reflected, “It's just tremendous to be able to give students these kinds of lines on their resume and these kinds of substantive experiences.” All of this occurred due to the support of the Honaker Family Public Service Endowment sponsoring guest speakers to inspire students and the support of Covenant College in encouraging in-depth, relevant academic collaboration between faculty and students.

Connect with us