OT: Are we on the cusp of an “A.I. bubble?”

[XXtwindad]

Read an interesting op-Ed in the NYT the other day. It’s fair to say the author is not bullish on the future of A.I., which she writes has been severely overhyped. The subject has always interested me, but it’s above my pay grade.

https://www.nytimes.com/2024/05/15/o...ated-hype.html

“The biggest question raised by a future populated by unexceptional A.I., however, is existential. Should we as a society be investing tens of billions of dollars, our precious electricity that could be used toward moving away from fossil fuels, and a generation of the brightest math and science minds on incremental improvements in mediocre email writing?

We can’t abandon work on improving A.I. The technology, however middling, is here to stay, and people are going to use it. But we should reckon with the possibility that we are investing in an ideal future that may not materialize.”

Anyone more well-versed on the subject?

[Turkle]

I work in an industry that is fully on-board with the AI fad, which is utterly embarrassing.

Let me tell you as clearly as I can: "AI" as it presently exists is, outside of a very few niche applications, at best an interesting toy, and at worst a dangerously unreliable solution. Its most useful applications appear to be plagiarism and the production of unbelievable amounts of useless trash content at scale, all at the cost of devastating power consumption and all the attendant consequences for climate change.

The hype around "AI" (I can't help putting it in quotes, as there is nothing even remotely similar to human intelligence happening here) is exactly like the hype around the Metaverse a year or so ago. Don't remember anything about the Metaverse? Of course you don't, it was complete BS. God, remember all the idiocy surrounding "The Blockchain" and "NFTs?"

The ridiculous hype surrounding the Metaverse and "AI" can both be explained fairly simply. The largest tech companies have already pretty much shot all their best bullets. The information revolution is complete. But Wall Street still demands explosive growth. So, in the absence of any actually useful new technologies, they are reduced to hyping inane fads.

"AI" is a solution looking for a problem, and it will be largely forgotten in a year or two.

[C40_guy]

Originally Posted by XXtwindad

Anyone more well-versed on the subject?

Pretty much everyone, as compared to the writer.

She did get one thing right:

"The reality is that A.I. models can often prepare a decent first draft."

AI is in the middle of a hype curve, not unlike the tulip mania upswing of many years ago. The marketing hype would have you believe that Ai, and specifically generative AI, is the answer to all the worlds problems, challenges and ills.

Um, no.

And by the way, I'm no AI fan boy. I do have some hands-on experience....I built my first expert system in 1986. On a PC.

As a tool, genAI can be quite useful, just like a reciprocating tool. Use it for the right task, and life is good. Use it for the wrong task, and you're going to make a mess.

GenAI is just a tool.

[CMiller]

I use ChatGPT every day to help me write R code as an economic consultant, it literally makes me 10x more productive when I have a complex programming task.

It's not hype for me.

[CMiller]

I wouldn't consider data analytics and coding to be niche industries, and nearly every data scientist/analyst/programmer my age I know uses it (Mid-30s).

You can be cautious about its potential, but so confidently putting down its utility is short sighted when it's already making real improvements.

[charliedid]

I have no idea.

[prototoast]

Originally Posted by CMiller

I use ChatGPT every day to help me write R code as an economic consultant, it literally makes me 10x more productive when I have a complex programming task.

It's not hype for me.

If you're an economic consultant and live where you say you do then there's a 50% chance that I had your exact exact job about a decade ago, and can finally say "back in my day..." we had to google how to code the stuff we didn't understand, not any of this chatGPT stuff.

Anyway, I think computer programming is an area where AI is really good (it kind of makes sense that a computer ought to be good at telling you what to tell a computer so the computer does what you want it to do). Now, my economic work involves much less coding and much more subject matter analysis, and the use of generative AI is both prohibited by my organization and functionally useless.

It's good where it's good, and I think it has a lot of productivity-enhancing capabilities, but I am also skeptical that it will totally transform society in the short-to-medium term. So I guess maybe I'm aligned with the median expectations about AI?

[Shane4XC]

Originally Posted by Turkle

I work in an industry that is fully on-board with the AI fad, which is utterly embarrassing.

Let me tell you as clearly as I can: "AI" as it presently exists is, outside of a very few niche applications, at best an interesting toy, and at worst a dangerously unreliable solution. Its most useful applications appear to be plagiarism and the production of unbelievable amounts of useless trash content at scale, all at the cost of devastating power consumption and all the attendant consequences for climate change.

The hype around "AI" (I can't help putting it in quotes, as there is nothing even remotely similar to human intelligence happening here) is exactly like the hype around the Metaverse a year or so ago. Don't remember anything about the Metaverse? Of course you don't, it was complete BS. God, remember all the idiocy surrounding "The Blockchain" and "NFTs?"

The ridiculous hype surrounding the Metaverse and "AI" can both be explained fairly simply. The largest tech companies have already pretty much shot all their best bullets. The information revolution is complete. But Wall Street still demands explosive growth. So, in the absence of any actually useful new technologies, they are reduced to hyping inane fads.

"AI" is a solution looking for a problem, and it will be largely forgotten in a year or two.

I’m not sure what industry you’re in but AI has way more consequences than just plagiarism and creating junk.

A couple of side projects I’ve personally touched with AI:
Anomaly detection for intrusion detection
Object detection within geospatial images

And those projects were wildly successful. Everyone sees language models and thinks that’s all AI encompasses.

[verticaldoug]

I think you are using the wrong metric. When you say 'Bubble' you are really speaking about valuation in an economic sense. In 1999, the internet craze was a bubble, and the market did crash and a lot of bad ideas went out of business. However, the internet proved really useful and has continued to grow to valuations which are really quite staggering when compared to 1999. The Internet was really useful before the iphone (or smartphone) in general. But the ability to have access to the world wide web while mobile has really changed things.

So is AI a bubble. Yes, probably since the same lack of discipline for investing in any hairbrained startup seems to be upon us. But will AI prove extremely useful and is here to stay? Without a doubt. In 15 years, will we look back and be staggered on how far we have progressed, we are just beginning to see the build out of the software stack which will really improve the useability of GenAI. Just like in 1999, when the crash comes for AI, you need to take a deep breath and buy some of the survivors because those will continue to grow.

For CMiller, I also use chatGPT for coding in python. It takes the drudgery out of a lot bulk programming, and allows you to spend more time the functionality you are hoping to achieve. It's great.

For prototoast, out of the box, chatGPT is not useful for economics, but you need to think about how and why you use it. If you are using it to just generate text from prior knowledge, it's as worseless as most economic opinions. If you add additional knowledge to 'ground' the system in facts, curtail its use of 'prior knowledge' in the system, I think you can get a really useful co-pilot. I know what Salesforce is doing with their Einstein Co-pilot to integrate it with their CRM, and you get some real pick-ups in productivity. In a general sense, this is also what Perplexity is trying to achieve with search+AI with transparent use of citations for results.

I suspect for most users, they write a lazy short prompt and expect a miracle. They get a disappointing result and say AI is hype. But if you actually spend time learning how to instruct in detailed concise language, you may surprise yourself.

The biggest problem is many 'experts' aren't really experts and they are just talking garbage. As someone mentioned Metaverse, yeah many of the so-called metaverse experts on social media are now experts on prompting etc.

Generative AI is just one aspect of AI. Pure generation because of the sampling probability inherent in the models, will always be fraught with hallucinations. However, for models grounded with additional curated factual information, it can be great.

Predictive AI, scoring models, image recognition etc, are all useful in their own right. Shane mentioned some of these in his post.

The big question is will these LLMs lead to AGI. I doubt it. I believe Yann LeCunn when he says we need to find another way.

Arthur C. Clarke said, “Any sufficiently advanced technology is indistinguishable from magic”. To most people this is just magic.

[Davist]

I'm in the power end of the data center business, certainly money is being spent at an unprecedented rate. Is it a bubble? perhaps, but as above, there are a lot of useful things being done across a wide range of fields with it. I view it as an extension of machine learning, which at its basic form helps automate things like the spam detector on your email. It's very narrow based, and I share the view that "general" version of AI is a long ways off and may not even be possible. I do understand the plagiarism argument, especially with writing and art images, it's hard to view otherwise. It's interesting this has been a topic in science fiction, like in Dune where they've sworn off AI and use human computers and some other being hopped up on "spice" (sea creature or something) for the navigators.

[CMiller]

Originally Posted by prototoast

If you're an economic consultant and live where you say you do then there's a 50% chance that I had your exact exact job about a decade ago, and can finally say "back in my day..." we had to google how to code the stuff we didn't understand, not any of this chatGPT stuff.

Anyway, I think computer programming is an area where AI is really good (it kind of makes sense that a computer ought to be good at telling you what to tell a computer so the computer does what you want it to do). Now, my economic work involves much less coding and much more subject matter analysis, and the use of generative AI is both prohibited by my organization and functionally useless.

It's good where it's good, and I think it has a lot of productivity-enhancing capabilities, but I am also skeptical that it will totally transform society in the short-to-medium term. So I guess maybe I'm aligned with the median expectations about AI?

I agree with all of this.

For programming it is absolutely useful. I switched my main analytic program from Stata to R within a few weeks thanks to AI. ChatGPT isn't prohibited here, but every single sentence we send is recorded to make sure we do not share client info. You can click a setting for ChatGPT to not record your inputs to use it for their model building, but we take extra precautions.

It's easy enough to feed dummy data: "I have a sheet of data with row names: var1, var2, var3, listed in wide/long/whatever format." It is useful when you are applying a new statistical technique, or creating client-ready formatted documents that would be tedious to figure out step by step.

And yes, I am a new entry into on one of those firms that are across the street from eachother, small world!

[el cheapo]

What's scary about AI is the ability to mimic voices and add them to video and have complete control of whatever message you want to send. This is going to be a nightmare for national security.

[verticaldoug]

Originally Posted by Davist

It's interesting this has been a topic in science fiction, like in Dune where they've sworn off AI and use human computers and some other being hopped up on "spice" (sea creature or something) for the navigators.

How can you forget the first season of Picard? AI is an existential threat.

Or Vger from Star Trek the Motion Picture?

[prototoast]

Originally Posted by verticaldoug

For prototoast, out of the box, chatGPT is not useful for economics, but you need to think about how and why you use it. If you are using it to just generate text from prior knowledge, it's as worseless as more economic opinions. I think if you add additional knowledge to 'ground' the system in facts, curtail its use of 'prior knowledge' in the system, I think you can get a really useful co-pilot. I know what Salesforce is doing with their Einstein Co-pilot to integrate it with their CRM, and you get some real pick-ups in productivity.

I wouldn't rule out the possibility that someone could use it more effectively than me, but there are a couple of big obstacles.

1) I am often dealing with nonpublic information, and I cannot input that knowledge into the system because that is sharing the information beyond where it is authorized to be shared (it's conceivable in the future each organization may have its own closed instance of chatGPT but we're not there yet).

2) My current work is largely in tax policy, in which there is significant intersection of law and economics. There have been plenty of documented instances where generative AI has simply made up laws or legal citations, and in these cases, that's negative value add. If, for example, I am trying to explain why corporate tax receipts shifted in response to a change in law, it's really, really important I've got all the details of the law changes right. There are some folks out there looking to develop special implementations for this purpose, but this is a pretty huge endeavor on its own, and from what I've seen, they're not ready for primetime yet.

3) I'm often working on very niche topics, and in some cases I have plausibly read all there is to read on a topic. For very big topic areas, I could imagine a prompt along the lines of "summarize the literature on the economic consequences of the collapse of Lehman Brothers" might yield something informative, but "summarize the literature on the economic consequences of consumers not paying excise taxes on fishing rods purchased directly from foreign sellers" yields about as much information as if I asked you that question and you just rambled off whatever came to mind.

[verticaldoug]

1) I am often dealing with nonpublic information, and I cannot input that knowledge into the system because that is sharing the information beyond where it is authorized to be shared (it's conceivable in the future each organization may have its own closed instance of chatGPT but we're not there yet).

I know several large firms which have their own instances of OpenAI, Google which are unique to them for use of in-house information.


2) My current work is largely in tax policy, in which there is significant intersection of law and economics. There have been plenty of documented instances where generative AI has simply made up laws or legal citations, and in these cases, that's negative value add. If, for example, I am trying to explain why corporate tax receipts shifted in response to a change in law, it's really, really important I've got all the details of the law changes right. There are some folks out there looking to develop special implementations for this purpose, but this is a pretty huge endeavor on its own, and from what I've seen, they're not ready for primetime yet.

Yes, not allowing the LLM to use 'prior knowledge' but instead give it data retrieval of an indexed legal database where it can search and give transparent traceable citations which can be checked. There is a start-up called HarveyAI which is trying exactly this. I'd expect someone like WestLaw to indice their libraries, and build a retrieval augmentation system for the LLMS. I don't know why this can't be done. As long as you have transparency of citations lazy people actually check.

The problem is less the AI and more the lazy user.