ChatGPT use declines as users complain about ‘dumber’ answers, and the reason might be AI’s biggest threat for the future::AI for the smart guy?
Back in my day, we used to call ‘prompt engineering’ ‘asking a question’.
They got to have a special termonology because what they do is oh so special. Some AI users act like they’re Louise Banks from the movie Arrival cracking the code to an alien language or something. And I don’t think it’s far fetched to assume they’re often from the same breed who had NFT monkeys as their twitter pfp about 18 months ago.
Blockchain > Crypto > NFTs > LLMs > whatever’s next.
These people will always be sniffing around for the next big thing to oversell and fleece their audience.
Its more than because half the time it doesn’t even answer the question.
It’s getting worse based on the feedback unfortunately, the need for safety and lack of meaningful deliberation towards how AI companies should operate and what should and should not be done has led Sam and co to be indesicive towards doing anything. Alongside the “morality” of the thing being hyjacked has lead to other AI’s performing better… lead by x employees of OpenAI, with actual bound morals and not inherently relying on user input to train future models, this will be the path forward, this will lead to safe and controlled integration.
I guess at the core of this, we are afraid of ourselves. We are afraid that the worste of humanity outpaces the better parts, that the inputs and training aren’t altruistic but are more pointedly “bad” or “wrong”, and thus leading to “harmful”, whether through misinformation, lies, or fabrications.
I hope we find a way to do better. I’m still excited for the future of AI, I mean crap, I’m closer to having a family doctor that’s a robot then I am to a real human doctor.
I guess at the core of this, we are afraid of ourselves. We are afraid that the worste of humanity outpaces the better parts, that the inputs and training aren’t altruistic but are more pointedly “bad” or “wrong”, and thus leading to “harmful”, whether through misinformation, lies, or fabrications.
Is there any reason not to be afraid? I think you could say that Tay was essentially the same idea a few years back and it took like 48 hours loose on the internet for it to spout literal Nazi (1930s-40s German NSDAP) rhetoric. Besides that being a PR disaster - if “AI” is only getting stronger and more integrated into human life and society, that can be pretty problematic.
Removed by mod
They could make it paid only today, and it’d be instantly profitable. Most free users would transition to a free alternative, but the corporate world would easily pay for use. So would some power users. But I’m sure they are making good money with all the API use anyways, the free access is a cheap way to get mass testing and training data.
When reality catches up to marketing
Nonsense. Less people are using it because there are viable alternatives and the broader novelty has worn off.
I use it every day in my job and the quality of answers only drops off when prompts are poorly crafted.
By and large, the average user doesn’t understand the fundamentals of prompt engineering.
The suggestion that “answers are increasingly dumber” is embarrassing.
Unfortunately I don’t agree with you. Different things have changed over time:
- For chatgpt 3.5 they moved to a “lighter” and faster (distilled) version, gpt-3.5-turbo. Distillation came with a performance price, particularly on advanced and less common cases.
- newer chatgpt-4 versions have likely been “lighten” for performance reasons
- context has been halved for chatgpt-4 on webui, meaning that the model forget more easily and can use half information to create text
- heavy control has been implemented on jailbreaking and hallucinations, that results in models less prone to follow complex instructions (limiting prompt engineering) and that prefer simplified answers than providing wrong ones (overall decreasing the chance of getting high quality answers).
All these changes have made working with gpt less pleasant, and more difficult for very advanced and specialized case, particularly with gpt-4 which at the beginning was particularly good.
This was really enlightening. Do you have some articles that elaborate? ☺️
Regarding 3.5 turbo you can check the documentation, the old 3.5 models are defined as “legacy”. Regarding max number of tokens of gpt-4 you can try yourself. It used to be >8k, it is now >4k from webui.
There is a talk from openai cio (if I recall correctly) where he describes that reinforcement learning from human feedback (rlhf) actually decreased performance of the models when it comes to programming. I cannot find it now, but it is around on YouTube.
The additional safeguard against jailbreaking, it is what OpenAI has been focusing the past months with heavy use of rlhf. You can google official statements regarding “safety” of the model. I have a bunch of standard pre-prompt I have been using to initialize my chats since the beginning, and with time you could see how the model followed the instructions less strictly.
Problem with openai is that they never released exact number of parameters they are using and detailed benchmarks. And benchmarks you find online refer to APIs that behave differently than the chat webui (for instance you have longer context, you set temperature and system prompt, they are probably even different models, who knows… All is closed)
Measuring performances of llm is pretty tricky, minimal changes can have big effects (see https://huggingface.co/blog/evaluating-mmlu-leaderboard), and unfortunately I haven’t found good resources to properly track chatgpt performances (from web ui) over time, across iterations
Thank you for the detailed reply 👍🏻
I use it daily too and haven’t had any of the issues I see written about it
deleted by creator
Article talks about the potential of AI cannibalism were it is now learning from data that it (or other AI) has generated.
Does ChatGPT use modern data I was under the impression that it’s most modern dataset was a few years old
You are using a free version.
ChatGPT4 and free Bing(ChatGPT) uses recent data
As long as it continues to do my resumes for me that’s all I need lol.
error loading comment
Surely the rampant server issues are a big part of that.
OpenAI have been shitting the bed over the last 2 weeks with constant technical issues during the workday for the web front end.
Why is it relevant what Peter Yang - Roblox product lead and enthusiastic child labor exploiter - tweets about it? Let me guess he’s a prompt engineer?
deleted by creator
It definitely got more stupid. I stopped paying for plus because the current GPT4 isn’t much better than the old GPT3.5.
If you check downdetector.com, it’s obvious why they did this. Their infrastructure just couldn’t keep up with the full size models.
I think I’ll get myself a proper GPU so I can run my own LLMs without worrying that they could stop working for my use case.
GPT4 needs a cluster of around 100 server-grade GPUs that are more than 20k each, I don’t think you have that lying around at home.
I don’t, but a consumer card with 24GB of VRAM can run a model that’s about as powerful as the current GPT3.5 in some use cases.
And you can rent some of that server-grade hardware for a short time to do fine-tuning, which lets you surpass even GPT4 in some niches.
deleted by creator
I’ve definitely seen GPT-4 become faster and the output has been sanitized a bit. I still find it incredibly effective in helping with code reviews where GPT-3 was never helpful in producing useable code snippets. At some point it stopped trying to write large swaths of code and started being a little more prescriptive and you still need to actually implement snippets it provides. But as a tool, it’s still fantastic. It’s like a sage senior developer you can rubber duck anytime you want.
I probably fall in the minority of people who thinks releasing a castrated version of GPT is the ethical approach. People outside the technology bubble don’t have a comprehension of how these models work and the capacity for harm. Disinformation, fake news and engagement algorithms are already social ills that manipulate us emotionally and most people are too technologically illiterate to see how pervasive these problems are already.
I was talking about it a month ago - others made fun of me… 😂
