Scientists from our Faculty are working on the Polish version of ChatGPT. It will be trained with material in Polish and data related to the Polish socio-cultural context. The first test version is to be published in the first half of next year.

The research is conducted by a team from the CLARIN-PL scientific consortium at the Department of Artificial Intelligence, Faculty of Information and Communication Technology.

– Our university's strategy includes research related to artificial intelligence in the first place as the main direction of development. – It should be remembered that artificial intelligence can have various applications, and what other institution, if not the university, should strive to make them as positive as possible – says Prof. Tomasz Kajdanowicz, head of the Department of Artificial Intelligence.

Artificial intelligence is being developed inter alia in the directions of large language models. Recently, ChatGPT, which was launched in December 2022, has been breaking popularity records. It is a generative language transformer developed by the company OpenAI. It is theoretically able to answer every question, while maintaining high linguistic accuracy. It is therefore used in education, business and everyday life.

Large language models

Scientists from Wrocław University of Science and Technology have been researching language technologies for many years as part of the CLARIN-PL project. They developed research infrastructure used mainly in the areas of humanities and social sciences. Over the last four years, they have been working on a wide variety of general and natural language processing databases.

– This is how we entered into what is now a symbol of artificial intelligence, i.e. large language models. We also came to the conclusion that it is necessary to create a large Polish language model, which in the future would be the basis for developing such solutions as Polish ChatGPT – explains Prof. Maciej Piasecki, coordinator of the CLARIN-PL scientific consortium.

As part of the planned research, scientists want to collect all available linguistic resources and knowledge regarding the construction of large language models and develop a solution that will be available to all interested people.

– At this moment, language models are beginning to shape the language we use. More and more content is generated at the user's request, and sometimes even in their place. Therefore, we would like our model to well reflect the specificity of the Polish language and have a positive impact on it, – emphasizes Prof. Maciej Piasecki. “It is a huge challenge, which is why we want to take the initiative to create a consortium, including as many scientific entities and private companies as possible. We are already cooperating among others with the Ministry of Digitization and the Information Processing Centre – National Research Institute – he adds.

The specificity of the language

The development of a „Polish" version of ChatGPT is important because currently the solution created by OpenAI still does not cope well with many elements related to the Polish language.

– We suspect that ChatGPT wasn't trained on enough Polish materials as compared to other languages. Therefore, there is a high chance that when preparing answers, it overwrites certain knowledge about Polish culture, customs and facts with data from other languages. During the tests, we noticed that this observation applies especially to Polish culture and history, and there are also some grammatical and stylistic errors, – explains Dr. Jan Kocoń from the Department of Artificial Intelligence. – It is in our interest to control this process and have control over information related to our country – he adds.

Currently, the Wrocław Networking and Supercomputing Centre is completing large research and development infrastructure that will be used to develop a large Polish language model. The key element will be one of the first supercomputers in our country specialized in natural language processing and artificial intelligence.

– We are buying equipment worth almost PLN 130 million. It consists of, among others: 300 H100 graphics cards used to train deep neural networks, over 30 petabytes (one petabyte is a thousand terabytes) of space on hard drives and a petabyte of RAM,” says Dr. Jan Kocoń. – We didn't just idly wait for this equipment to come to us. We already have collected almost 300 gigabytes of plain text from various sources and this number is constantly growing. We also employ 60 people who prepare instructions for this system, so that it can be tuned to implement our language model. You can join the research now – he emphasizes.

The scientists plan to make the first version of the program available for open testing in the first half of next year.

Not only ChatGPT

Research on large language models is not the only project related to the use of artificial intelligence that our scientists from the Faculty of Information and Communication Technology are currently working on.

Last week they received a grant of over PLN 1 million for the development of a tool which facilitates the searching for court decisions. It will be carried out in cooperation with two universities from France and Great Britain, and the work will be led by Prof. Tomasz Kajdanowicz.

The funding comes from the Open & Re-usable Research Data & Software (ORD) competition run by the CHIST-ERA network, which supports research in the field of information and communication technologies.