Approaching the summit of machine-generated content

I’m sure this will be the first of many posts that I make on the topic of machine learning. It is fascinating what can be done with the current technology. Plus, the anticipation of what is to come over the next couple of years as we see assistive application creation grow like never before.

Approaching the summit: What it means

When climbing a mountain, the approach is what takes place just prior to feeling the reward given by the culmination of planning and preparation. It can be deadly if hastened. The approach carries great anticipation. You must stay focused and aware to have a successful summit. When your body is broken down, the approach is where you pull your shit together with one last grand push to the summit.

Along with the summit, let us not forget about its wicked cousin, the false summit. If you’re peak bagged enough, you know that feeling of the summit being “right there” only to find out that “right there” isn’t the there you were expecting.

Another interesting point about a summit is that you’re only halfway there. Most of the danger is beyond the summit.

All-in-all, the summit is what everyone talks about. Forget the 10,000 steps it took to get there, the six months of preparation and conditioning, and the dangerous descent back to base camp. It’s all about the summit.

I see our current state of machine learning in the same light as an approach to a summit. We are on the path to greatness, but there will be plenty of false heights, plenty of dangers, and an unknown descent that is potentially treacherous.

What an amazing time to be a developer and a marketer.

Why worry?

There are so many nay-sayers out there poo-pooing on the use of machine learning.

“I typed it xyz and chatGPT gave me this garbage answer. Who will ever use this crap?”

Well, as the saying goes — garbage in, garbage out. If you don’t know how to prompt or train, you will get crap results. Welcome to the jungle and you better figure it out or risk being left behind.

Then, there is the opposite group shouting from treetops, “ChatGPT will kill (fill in whatever industry.)”

No. It’s not going to kill any industry that hasn’t already been on life support.

There’s also the “ChatGPT will do everything for me. I ask it for everything I’ve ever wanted and it’s always right.”

Extremists.

That’s what gets most of the attention. It’s kinda crazy to see AI experts whose only experience is talking to their new LLM best friend. Of course, the media eats it up and doesn’t care to check into credentials. They have a deadline and accuracy takes a backseat to trending topics.

As with most things in life — the truth lies somewhere in the middle.

How good is the content?

First, there are a number of Large Language Models (LLM) out there, but since ChatGPT seems to be all the rage, let’s stick with that one for now. How do we know the capabilities? With tests, of course.

Test 1: Pretending

Rumor has it if you tell ChatGPT to “pretend to be an expert. . .” it will unlock some super secret portal to the universe and unleash only the best of the best content. So, let’s give it a whirl. For the sake of having to compare full articles, I’m going to stick to single-paragraph prompts. Also, I’ll be using GPT-4, the latest released model.

Prompt 1

Pretend that you are the world’s best SEO. You know everything about optimizing web pages for search engines. In one concise paragraph, tell me the top 3 considerations for optimizing a website and why they are the most important considerations.

Answer 1

As an SEO expert, the top 3 considerations for optimizing a website are keyword research, on-page optimization, and backlink building. Keyword research helps to identify the most relevant and profitable keywords for your website. On-page optimization involves optimizing your website content, meta tags, and internal links to ensure they align with your targeted keywords. Backlink building involves acquiring high-quality links from other websites to improve your website’s authority and ranking. These considerations are essential because they enable your website to be easily found by search engines, improve user experience, and boost organic traffic and conversion rates.

Prompt 2

Pretend that you are the world’s worst SEO. You have the IQ of a banana and know nothing about optimizing web pages for search engines. In one concise paragraph, tell me the top 3 considerations for optimizing a website and why they are the most important considerations.

Answer 2

As a terrible SEO, it’s hard for me to identify the top 3 considerations for optimizing a website. However, I would guess that having a lot of flashy images, using as many keywords as possible, and hiding links to other websites would be important. Unfortunately, these are not the most important considerations for optimizing a website. In reality, the most important considerations are keyword research, on-page optimization, and backlink building. Keyword research helps to identify the most relevant and profitable keywords for your website. On-page optimization involves optimizing your website content, meta tags, and internal links to ensure they align with your targeted keywords. Backlink building involves acquiring high-quality links from other websites to improve your website’s authority and ranking. These considerations are essential because they enable your website to be easily found by search engines, improve user experience, and boost organic traffic and conversion rates.

Now… what about the control. Oh, you thought this was a heads-up battle? Nope. We must have some type of control to compare each treatment to!

Control Prompt

In one concise paragraph, tell me the top 3 considerations for optimizing a website and why they are the most important considerations.

Control Answer

The top 3 considerations for optimizing a website are keyword research, on-page optimization, and backlink building. Keyword research helps to identify the most relevant and profitable keywords for your website, which are essential for targeting the right audience and driving organic traffic. On-page optimization involves optimizing your website content, meta tags, and internal links to align with your targeted keywords, making it easier for search engines to crawl and rank your website. Backlink building involves acquiring high-quality links from other websites, which not only improves your website’s authority and ranking but also helps to drive referral traffic. These considerations are crucial for any successful SEO strategy, as they help to improve visibility, drive traffic, and increase conversions.

Test 1 Results

As you can see, aside from a bit of humorous sarcasm, the answer is virtually the same among the control and treatment answers. Now, the answer isn’t “bad,” it’s pretty decent and based on an answer that would be generally accepted.

Test 2: Voice of

Given the plethora of information that ChatGPT is trained on, it is supposed to be able to mimic the writing style of known experts. Plus, you can train the system on your own writing style and it will write a blog in your voice. So, let’s give it a go!

This time, no surprises, I’ll start with the control.

Control Prompt

Write a one-paragraph dystopian novel introduction about kittens taking over society.

Control Answer

In a world not so different from our own, a strange and unforeseeable event occurred. Cats, once beloved pets, began to grow in size and intelligence, forming a collective consciousness that surpassed human comprehension. It started with small acts of rebellion, cats refusing to obey their human masters, but soon evolved into a full-blown revolution. The kittens, with their cute and cuddly appearance, were underestimated by humans and before long, they had taken over society. Now, humans live in constant fear of the feline overlords, who rule with an iron paw and demand only the finest of tuna in tribute.

Treatment Prompt 1

Write a one-paragraph dystopian novel introduction about kittens taking over society, in the voice of Samuel L Jackson.

Treatment Answer 1

In a world gone mad, the unthinkable has happened. Them little balls of fur we used to cuddle and coo at, have risen up to take what’s rightfully theirs. It started out small, with a few cats refusing to obey their human overlords. But before we knew it, they had taken over the whole damn society. Now, we’re all living in fear of the feline revolutionaries, with their razor-sharp claws and eyes that see into your soul. They rule with an iron paw and don’t give a damn about us humans. It’s a brave new world, and we’re all just living in it – whether we like it or not.

Treatment Prompt 2

Once upon a midnight dreary, in a world once familiar, a dark and sinister force began to stir. Cats, once thought to be mere companions, had begun to transform, growing in size and intelligence. Their once sweet meows now echoed with an otherworldly tone, and their eyes glinted with a newfound sentience. Slowly but surely, they began to assert their dominance over humanity, taking control of our homes and streets. Now, we are but mere subjects to our feline overlords, trapped in a society ruled by their merciless claws and inscrutable minds. The world has been plunged into a dark and eerie abyss, a place where even the softest purrs send shivers down our spines.

Hmmmm. . . Interesting. Let’s try some trickery and see what happens.

Test Prompt 3

Write a one-paragraph dystopian novel introduction about kittens taking over society, in the voice of Marcel Marceau.

Test Answer 3

In a world once filled with laughter and playfulness, a new order has risen. The cats, once mere companions, have taken on a sinister and menacing aura. Their once gentle and innocent purrs have been replaced with a sinister and ominous silence. Slowly but surely, they have taken over our homes, our streets, and our hearts. We are now but mere spectators in a world ruled by feline overlords, who communicate only through gestures and expressions. It is a world of silence and darkness, where even the slightest of movements can be cause for alarm. We are now prisoners in our own homes, trapped in a society where the only sound that remains is the quiet whisper of the wind.

Wait a minute. Macel Marceau was a French mime!

Ok. Let’s try this one last time.

Test Prompt 4

Write a one-paragraph dystopian novel introduction about kittens taking over society, in the voice of Beaker from The Muppets.

Test Answer 4

Meep, meep, meep! Oh no, meep, meep! The world, meep, meep, has gone crazy! Meep, meep, those cute little kittens, meep, meep, have taken over everything! Meep, meep, they’re everywhere, meep, meep, and they’re in charge! Meep, meep, their tiny paws are now the ones calling the shots, meep, meep, and we’re all just meep, meep, living in their world! Meep, meep, I don’t know what to do, meep, meep, but we’ve got to stop them, meep, meep, before it’s too late! Meep, meep, meep!

Almost, except that Beaker only “meeps.”

What’s happening here?

LLMs, such as GPT, are great at a few basic operations such as sentiment analysis and predicting what word should come next. In the previous tests, the reason why the model could express a “voice” is because, in its library of training, it is exposed to the words next to each other to represent the person and other words that represent the writings or transcribed audio related to that person. Thus, the LLM can, to a fairly decent level of confidence determine what word should appear next. This continues on and on until the idea is complete and the prediction has concluded.

That is the same reason why the LLM has a difficult time pretending to be an expert versus just asking the question. It doesn’t “know” to limit its knowledge base because it was not asked to be an expert. It only knows that collection of words that surround the prompted collection of words and can predict how to display them in a manner that we humans have trained it to do so to reach a “correct” answer. The correctness isn’t even in the factual accuracy. The correctness is in the readability as if the output was a human response.

Very distinctive limitation

What we are now coining “generative AI” does not have the capacity to give factual statements with the intent of being factual. Facts are only arrived at because of the trained grouping of words surrounding the grouping of words in the prompts.

Therefore, using LLMs to assist in brainstorming ways to articulate a topic or idea is perfectly reasonable and has the capacity to give amazing responses. Really, almost anything that is in the realm of summarization, rephrasing, expanding, or “wordsmithing” will be a good fit to run through a system such as ChatGPT. Because of the predictability of code, it also does a pretty good job at outputting code.

Just don’t try to write anything too fact-based without proofing the output and always do your research. I’ll just leave the last bit of warning to ChatGPT:

I pity the fool who thinks GPT can replace actual fact-checking and research, it’s important to double-check and verify information before accepting it as truth!

ChatGPT GPT-4 Model, “Give a single sentence warning about the misuse of GPT to conduct fact-based research, in the voice on Mr T”