Yes. It can converse perfectly normal in German. However when quizzed about German idioms it hallucinates them (in fluent German). Though that's the kind of stuff even larger models often have trouble with. For example if you ask GPT 4 about jokes in German it will give you jokes that depend on word play that only works when translated to English. In normal conversation Llama seems to speak fluent German
For Ancient Greek I just asked it (in German) to translate its previous answer to Ancient Greek, and the answer looks like Greek and according to google translate is a serviceable translation. However Llama did add a cheeky "Πηγή: Google Translate" at the end (Πηγή means source). I know little about the differences between ancient and modern Greek, but it did struggle to translate modern terms like "climate change" or "Hawaii" and added them as annotations in brackets. So I'll assume it at least tried to use Ancient Greek.
However it doesn't like switching language mid-conversation. If you start a conversation in German and after a couple messages switch to English it will understand you but answer in German. Most models switch to answering in English in that situation
Your findings are Amazing! I have used ChatGPT to proofread compositions in German and French lately, but it would have never occurred to me that I should have tested ability to understand idioms, which are the cherry on the cake. I’ll have it a go
As for Ancient Greek or Latin, ChatGPT has provided consistent translations and great explanations but its compositions had errors that prevented me from using it in the classroom.
All in all, chatGPT is a great multilingual and polyglot dictionary and I’d be glad if I could even use it offline for more autonomy
I have tried to use Llama3-7b and 70b, for Ancient Greek and it is very bad. I will test Llama 3.2, but GPT is great at that. You might want to generate 2 or 3 GPT translations of Ancient Greek and select the best sentences from each one. Alongside with some human corrections, and it is almost unbeatable by any human alone.
Not one of these, but I tried on a small, Lithuanian, language. The catch is what the language has complicated grammar, but not as bad as Finnish, Estonian and Hungarian. I asked to summarise some text and it does the job, but the grammar is not perfect and in some cases, at a foreigner level. Plus, it invented some words with no meaning. E.g. `„Sveika gyvensena“ turi būti *atnemitinamas* viso kurso *vykišioje*.`
In Greek, it's just making stuff up. I asked it how it was, and it asked me how much I like violence. It looks like it's really conflating languages with each other, it just asked me a weird mix of Spanish and Greek.
Yeah, chatting more, it's confusing Spanish and Greek. Half the words are Spanish, half are Greek, but the words are more or less the correct ones, if you speak both languages.
EDIT: Now it's doing Portuguese:
> Εντάξει, πού ξεκίνησα? Εγώ είναι ένα κigneurnative πρόγραμμα ονομάζεται "Chatbot" ή "Μάquina Γλωσσής", που δέχθηκε να μοιράσει τη βραδύτητα με σένα. Φυσικά, não sono um essere humano, así que não tengo sentimentos ou emoções como vocês.
