regardless of whatever else you think about AI, it’s a good idea to be very wary about security-sensitive apps that are written using LLMs.

there’s a self-hostable S3 replacement called RustFS, with development that leans heavily on LLMs.

they had a security vulnerabilty last year.

one LLM-written commit added a complete authentication bypass to the code. literally, you could just send rustfs rpc as the auth token instead of an actual token.

it was fixed in a pull request named fix: Prevent panic in GetMetrics gRPC handler on invalid input which claimed to fix a different auth problem and just coincidentally removed the hardcoded auth token.

security stuff is hard, and LLMs are prone to over-confidence. that’s a bad combo.