It's somewhat subtle and easy to miss, but one of the feature points raised in the video about the AIP action graph is that users can "govern LLMs, defining when they can act alone and where they need human input."
So while the demo does mostly show back-and-forth between the LLM and a user, I'm still somewhat worried about what the limits of that "action graph" are going to be and how much of that back-and-forth in the demo is actually mandatory, because I do get the impression that part of the feature set is going to be automated responses. Maybe (hopefully) I'm wrong.
But even if that action graph is taken away and the chat back-and-forth is the only interface, then consider one of your examples:
> When the user tells the LLM to send the recon drone
This is still a giant problem. Can the LLM send the recon drone and it's just instructed to wait for the user to tell it to, or (preferably) does the LLM have zero control over the recon drone and is the user sending it through a separate interface that is not connected to the LLM at all? To me from the demo, it looks to me like a user can just ask an LLM to perform that action. That's a big problem, the product shouldn't launch with that feature.
Because if the LLM has access to that API and if there's any opportunity for malicious input (image analysis/OCR, web searches, etc...), then that input can cause the LLM to skip authorization and send the recon drone without human approval.
> you should definitely be just as worried as humans augmented with the ability to more efficiently kill others, even if it's only with what amounts to an AI secretary.
You're right, and it feels a little weird to have the primary criticism be "they're not securely killing people with AI." But for every action you've described above I have to ask, "can a 3rd-party reprogram the AI and get it to lie about the result of that action or get it to take that action without human intervention?"
I'm worried about military drone strikes from an ethical perspective, I'm worried about AI security within the military from a "holy crap, who authorized this and why haven't they been fired?" perspective. I'm not sure I've seen a single large commercial deployment of an LLM wired to real-world systems where I felt like the company building that product had rock-solid security; so I'm doubtful that Plantir is going to be the first one.
Yes, I think dispatching the drone could be done in a safer way. E.g., by sending the user some button to click on to send the command the LLM suggests to the drone. They have the confirmation messages saying what action was performed, but that's still problematic for the reasons you mentioned.
So while the demo does mostly show back-and-forth between the LLM and a user, I'm still somewhat worried about what the limits of that "action graph" are going to be and how much of that back-and-forth in the demo is actually mandatory, because I do get the impression that part of the feature set is going to be automated responses. Maybe (hopefully) I'm wrong.
But even if that action graph is taken away and the chat back-and-forth is the only interface, then consider one of your examples:
> When the user tells the LLM to send the recon drone
This is still a giant problem. Can the LLM send the recon drone and it's just instructed to wait for the user to tell it to, or (preferably) does the LLM have zero control over the recon drone and is the user sending it through a separate interface that is not connected to the LLM at all? To me from the demo, it looks to me like a user can just ask an LLM to perform that action. That's a big problem, the product shouldn't launch with that feature.
Because if the LLM has access to that API and if there's any opportunity for malicious input (image analysis/OCR, web searches, etc...), then that input can cause the LLM to skip authorization and send the recon drone without human approval.
> you should definitely be just as worried as humans augmented with the ability to more efficiently kill others, even if it's only with what amounts to an AI secretary.
You're right, and it feels a little weird to have the primary criticism be "they're not securely killing people with AI." But for every action you've described above I have to ask, "can a 3rd-party reprogram the AI and get it to lie about the result of that action or get it to take that action without human intervention?"
I'm worried about military drone strikes from an ethical perspective, I'm worried about AI security within the military from a "holy crap, who authorized this and why haven't they been fired?" perspective. I'm not sure I've seen a single large commercial deployment of an LLM wired to real-world systems where I felt like the company building that product had rock-solid security; so I'm doubtful that Plantir is going to be the first one.