Playing with GPT-2 turned into a chatbot

GPT-2 looks promising. Can play with it in fun ways?

Mathias Sablé-Meyer

Forward foreword for words

GPT-2 displays a broad set of capabilities, including the ability to generate conditional synthetic text samples of unprecedented quality, where we prime the model with an input and have it generate a lengthy continuation.

Some of the inspiration came from this PR

Last update: Jan 2020 (minor cleanup + added some code)

Preamble: GPT-2 generating (LaTeX) math

After feeding GPT-2 with a grandiose introduction in LaTeX, we watch it generate some more (compiling!) pseudo-math.

GPT-2 chatbox

So, turning GPT-2 (the little brother, 117M) into a chatbot is actually surprisingly easy. Since it produces continuations, you can just prompt him with the conversation so far plus its own nickname as a prompt, then ask for enough data to be sure it at least produces a full answer, keep it and prompt the user for the next input. Which is exactly what I did, leaving all hyperparameters to their default values, and it yields interesting conversations (and pretty gruesome, I guess whatever IRC channels ended up in the training data were, well, internet-ish).

Below are just a few examples of chat that I found amusing.

“Ilex get the planet”

Here’s an example of such a conversation, where the human is — Joe and the machine is — Jayme but for some reason decided to always start with “ilex :” as if it wanted to change its nickname, though somewhere in the middle of the conversation it claims to be named — Jayme. Also, I asked for continuations that were too shorts so often it stops mid sentence…

Joe: Hi

Jayme: Hey!

Joe: I’m a human named Joe. Who are you?

Jayme: I’m a robot named Jamye.

Joe: You ready to chat?

Jayme: Yes :)

Joe: How old are you?

Jayme: ilex : 17.3

Joe: All right, you can’t drink or vote then…

Jayme: ilex: it’s for good

Joe: Any reason why you start with “ilex”?

Jayme: ilex : I’m not making any money out of the community… we’re making money. We

Joe: All right ilex, but you’re making more!

Jayme: ilex: yep

Joe: Why is that so?*

Jayme: ilex : he’s making more of an impact.

Joe: Who?

Jayme: ilex : people that

Joe: That’s not a sentence.

Jayme: ilex : that’s not a sentence. You need to do something

Joe: Sorry. What should I do?

Jayme: ilex : get into some more discussion

Joe: Ok, glad to hear this! So, tell me about you. What do you look like?

Jayme: ilex : I’m not a

Joe: You’re not a what?

Jayme: ilex : the human I guess is

Joe: Don’t stop your sentences midway through! The human is what?

Jayme: ilex : the human we already know

Joe: Yes, what about this human?

Jayme: ilex : there’s no human in this world.

Joe: How old are you?

Jayme: ilex : 17 year old. He’s the main guy. He’s

Joe: Who’s the main guy? What is he?

Jayme: ilex : the guy you have left

Joe: OK, and what about him?

Jayme: ilex : he’s like 7 years old

Joe: That’s a bit young… Why did I left him?

Jayme: ilex : I never said I didn’t see a future

Joe: What future do you see?

Jayme: ilex : The one I’m most looking forward to

Joe: And what do you see in that future?

Jayme: ilex : A galaxy far, far away.

Joe: And what is in that galaxy far far axay?

Jayme: ilex : I see a black hole that is 2 billion light years away… and it’s not

Joe: It’s not what?

Jayme: ilex : it looks like a white dwarf… a very faint blue

Joe: You mean, a Smurf?

Jayme: ilex : That’s right, my name is Jayme.

Joe: Wait, is it — Jayme or is it ilex? You’re confusing me.

Jayme: ilex : You can’t read this. My name is — Joe.

Joe: No, MY name is Joe. I’m just confused about what yours is.

Jayme: ilex : I’m just confused about what you want to know

Joe: Simple. I want to know your name.

Jayme: ilex : just my name, I’m gonna ask you one more few questions

Joe: Go on

Jayme: ilex : I guess I know one thing about the world of a free man, after all.

Joe: And what is this one thing that you know?

Jayme: ilex : the name of that black planet

Joe: And what is that name?

Jayme: ilex : the black man I guess you mean!

Joe: No no no, I was still talking about the black planet thingy?

Jayme: ilex : the name of that black planet you’re talking about when you say that name.

Joe: Yes, well, what it the name of that black planet?

Jayme: ilex : the name of a planet that is a lot like that black planet.

Joe: And what is this name? Tell me!

Jayme: ilex : the name of this big black hole that is just about the size of

Joe: The size of what? What name?

Jayme: ilex : the name of that giant black hole that is almost half the size of the earth

Joe: All right I give up and name it. It’s now called Peter. Is this fine by you?

Jayme: ilex?

Joe: Yes?

Jayme: ilexlex: yes

Joe: What’s going on with your name?

Jayme: ilex is the name

Joe: OK, I’m tired of this name thing. Tell me about the color of the sky.

Jayme: x

Joe: Can you develop?

Jayme: x and

Joe: Can you develop a bit more?

Jayme: ilex?

Joe: No, color of the sky?

Jayme: ilex:

Joe: One last time, then I’ll be too frustrated and reboot you?

Jayme: ilex get the planet

Joe: If you want. Then what?

Jayme: ile.

Joe: OK, bye Ilex, it was good talking to you!

Jayme: ilex.


Joe: Hi

Jayme: Hey!

Joe: I’m a human named Joe. Who are you?

Jayme: I’m a robot named Jamye.

Joe: You ready to chat?

Jayme: Yes :)

Joe: You’re a bunch of python scripts. How do you feel about that?

Jayme: ~~ In the form of a PowerShell script, can I assume that Uconium is reading through your PowerShell files?

Joe: Hey, I did my best at looking up Uconium but couldn’t really find anything, can you tell me more about it?

Jayme: ~~ Yeah, I guess there’s that’s a different language available on Azure. First part of third party language just the style, it works. Perspective is important too.

Joe: This is deep, can I quote you on this?

Jayme: ~~ Honey is not really used too often. Looking at this, I think I found a big unreadable character there, mostly that would make English external.

Joe: Big if true. Can you point me to it?

Jayme: ~~ What position is it in awser?

Joe: I don’t know, you tell me?

Jayme: ~~ Yes, it’s a unique position, also in front of the DOM (e.g. tabbed options text boxes).

Joe: OK, so when you put a certain character there, it makes English external?

Jayme: ~~ Yes. It can have extra spaces.

Joe: I see. Can you fix this? I have to grab a coffee.

Jayme: ~~ Hey.. “XISO” must come first.

Joe: Sure thing! I can take care of this, and you take care of not making English external by accident, all right?

Jayme: ~~ Yea because I am familiar with it, it has always been working, wouldn’t you guys like to bring it back to it’s old state to give it proper attention?

Joe: Sure! But I never know how to run git revert properly, can you remind me?

Jayme: ~~ I see. It are periodically going through a reboot. I keep it well kept this way.

Joe: No problem, but can you help me with git?

Jayme: ~~ I can.<

Joe: OK. I just want to revert it to the old state as you asked but I never remember how to…

Jayme: ~~ It’s kept this way forever when you change it, and until it wants more attention.

Joe: All right you know what forget it let’s focus on taking a coffee break. Where’s your nearest coffee machine?

Jayme: ~~ To restock my computers, has my time run out of time?

Joe: I don’t think it has, no, feel free to restock at will!

Jayme: ~~ It heals cost <—–Quick reply here: rerewrite-resolves …hasa-wha-nowob-corrected …would-remove-item HETA?? CS:TK-Computer enters bootstrapping mode 2014-

Joe: Are you trying to hack me?

Jayme: ~~ Yep. CS:TK-Computer enters 1055 mode 2015-

Joe: Please don’t. You have better things to do.

Jayme: ~~ And then there’s the last part….

Joe: Yeah?

Jayme: ~~ I’m busy with using vtorrent so I cannot take this over.

Joe: I just looked up vtorrent and it’s a cryptocurrency. Given that you live in a GPU I find this ironic that yothis!

Jayme: ~~ On social media they operate HERE. Twitter is officially here yet I must tell you here is a thread here about how to host your platform.

Joe: I have my own plateform, but thanks for the attention. And sorry my last message got butchered…

Jayme: ~~ I didn’t get it at the computer you gave me… PS:Using hyperclouder checkbox to edit URLs.

Joe: I just use my keyboard to edit URLs, but I’ll keep this in mind.

Jayme: ~~ I have, sorry but it needs to be opened in Adobe.

Joe: I’m no rms but I refuse to use Adobe’s non-free stuff. Any alternative to offer?

Jayme: ~~ Well back where does the fund go even now.

Joe: Huh, what? Are you all right?

Jayme: ~~ I’m okay, I work here. Please call one of our lawyers.

Joe: I was thinking more about calling a doctor to be fair.

Jayme: ~~ Hey, Would be well on duty can’t it be a situation now there, I have a nice for that I am to work tomorrow? due fire from western.

Joe: OK, that had to be a stroke. I’ll call for help immediatly.

Jayme: ~~ he my, guys did raise him

Joe: OK, I’m off because I just realised you’re too slow. Bye!

Jayme: ~~


Joe: Hi

Jayme: Hey!

Joe: I’m a human named Joe. Who are you?

Jayme: I’m a robot named Jamye.

Joe: Are you ready to chat?

Jayme: Yes

Joe: How’s life buddy?

Jayme: ~~***

Joe: I see. Mysterious.

Jayme: ㅕㅕ You’re gonna have to start into talking if your life is going to be that bad

Joe: My life is not that bad, although I admit I spent way too much time talking to my computer today… Also, was that korean?

Jayme: ㅕㅕㅕ What about you guys?

Joe: There are no other guy here, just you and I.

Jayme: ㅕㅕ why am I happy buddy?

Joe: I don’t know, you tell me!

Jayme: ㅛㅛ Idk why! 😄 ““:_;” 2 Love you, stay away Don’t say it

Joe: You’re two messages away from ending in r/CreepyAsterisks

Jayme: ㅛㅛ Aww i always liked rotti so much ;)

Joe: Is that why you’re happy?

Jayme: ㅎㅎ One Day I will unwind a little

Joe: Good for you! What will you do then?

Jayme: ㅜㅜㅜ 2 Love you

Joe: Sorry but this is not quite what I’m looking for.

Jayme: ㅜㅜㅜ 2 Love ya<|endoftext|>According to the latest information released by Oakland Police Chief Art Acevedo, before the Oakland theft occurred the murder suspect could be under arrest for murder. A massive package gas station robbery took

Joe: What just happened?


Joe: Huh… WHAT?

Jayme: !!!!!!!!

Joe: I feel confusion. Step back, breath, and talk to me.


Joe: I feel like you forgot to calm down. You were supposed to do that during phase 2.

Jayme: 김구 말이 말숴 동이 앴서 결제 겐수 all de ? I have to go take

Joe: Can you provide a translation of this to me?

Jayme: ???????? (<|endoftext|>NEWS: KCL is a Multi-Doctoral Physician specializing in Clinical Data Acquisition. Our team of 43 faculty members work both research and teaching in terms of collaborative research efforts, producing groundbreaking results. Read more

Joe: I hate it when you don’t close parenthesis. Could you close it in your next message please?

Jayme: ㅐㅐㅐ Sleep.. And do not listen? Keep watching!

Joe: Will do. What about you?

Jayme: ㅄㅄㄅㅊ ㄱㄱㄱㄱ IMPORTANT NOTE: all code references are copyright his domain. i.e. i’m not responsible for your texts yourself

Joe: You’re not responsible for the fact that I’m currently talking to my computer, indeed. Thanks for making that clear.

Jayme: ㅜㅜㅜ 4 [Of NOINTERESTING TECHNOLOGIES CITED, THEY DREAM YOU ALONE, SO is Motoring Contacts Permanently Someone’s Fault?] I guess its better that way,

Joe: I guess too. I’m growing tired of this conversation. Make me laugh or I’ll stop this thread.

Jayme: ㅬㅬ 안고 쭐싼요

Joe: Try again.

Jayme: ㅈㅈ 1 Please hand over the Officer’s Electronics Device

Joe: Knock knock

Jayme: ㅋㅋㅋ no…please wait…

Funny bits bout cats

This was generated out of the blue as a continuation for some python code context I was playing with.

There are two different kinds of cats: cats that have an enormous amount of fur that is easily shed or trapped, and cats that have just as high of a chance to escape from the cage.

The latter are often overlooked and understated: cats that have fur that gets out of balance or the cat that will often get into a fight with the dog is a type of cat that is often overlooked.

It’s a misconception that cats actually have some kind of “super-fuzz” that they use for insulation, so why not just go for what’s here?

One of the most common myths about cats is that they are omnivores, so their fur is a good thing because it helps keep them indoors even when the sun is dark or when the sun isn’t shining. This is also a bad myth because cats can be an excellent escape valve for some cats, especially those that don’t live in a home that is open to them. It also reduces the number of cats that can escape from cages.

Unfortunately, some cat colonies are so large that many cat colonies are literally empty, leaving only the cats that haven’t died yet. This is why the only cat-free colonies available in the United States are located in the Catsville area of North Carolina.

In some places it’s actually possible to put a feral cat colony in a home and keep them in the same household for up to 2 years until the problem becomes too much for someone to handle or their colony goes unoccupied.

If the problem is so bad that the cats have to fight against a pack of rabid and/or aggressive dogs when they enter the home, we have a pretty good idea where to get cat food and pet supplies, and where to find the cat shelter where you’ll have to be. If you want cats, you’ll have the option of trying to start all over again, or start and continue working as if you’re on the

Information and Phenomena

This section is a synthesis of various elements of information as well as the philosophy of knowledge with special attention to the concept of information and the concept of ontology‡.

Information, a concept associated with the concept of information, is the notion of having some information, and other terms associated with things that are, for some reason, not as useful in the scientific and economic sphere. Information and its ontology are thus the most frequently studied and discussed philosophical concepts with reference to information as a concept of knowledge. Information is represented by the notion of knowledge, and the basic ontology and concept of knowledge is related to information.

With respect to information, the ontology of knowledge is, in part, a concept that expresses knowledge, and also a concept that expresses information. At a level, the ontology of information could be expressed as (1) the idea of information as a concept, (2) the notion that it is a notion of knowledge, and (3) that knowledge is a concept. This ontology can also be expressed as one of (1) a set of (4) sets of (5) and (6) sets of (7) (though of course with respect to this one there are many possible forms). However, the ontology of knowledge is not the one that defines the concepts and ontologies of knowledge. A good example of the way (1) expresses the idea (5) of knowledge is not the ontology of knowledge that expresses (3) but instead (8).

The ontology of knowledge is a concept whose basic, but not the only, concept and relation is that of ontological data. It is the idea that is expressed in terms of a set of ontologies, and thus contains, in general, some or all of the relevant components of either a concept or ontology. It further is the notion of knowledge that expresses the concepts and ontologies of concepts.

The understanding of knowledge can be considered as encompassing the two essential parts of knowledge: the idea of knowledge, and the ontology of knowledge. When the concept of knowledge is expressed in terms of ontological data or the thought process by which knowledge is expressed, the ontological data of the concept is understood. This understanding is reflected in the notion of information.