AI Hallucination Watch

Latest Posts

Opinion

Searching for a 3D Printer

If you read my last post you'd know I'm developing an enclosure as part of my thesis project. Enclosures are great - they keep adversarial events from destroying your security! Anyway, I don't have a 3D printer, CNC, or manufacturing hardware lying around so my first approach to manufacturing this enclosure was to outsource it to someone else. I Google'd "3D printing companies" and stumbled across Xeometry. They had a nice, easy to use, easy to get started UI.

One thing about my thesis is that I will be developing 2 security camera modules - 1) for use in testing indoor scenarios, and 2) for use in testing outdoor scenarios. The 2 units will use the same enclosure, raspbery pi model, camera model, and software, but they will need different internal hardware to overcome their unique enviromental challenges. For this reason, I need 2 copies of my enclosure (to enable simultaneous testing). I haven't put much thot into the materials I'd use but I heard PC-GF30 (polycarbonate plastic mixed with fiberglass) was a pretty good material. At one point Copilot told me this is what the Google Nest cameras were made of (not the best camera but a competitive analysis is not a component of this post). But, I was still open to options. I got quotes for Xeometry for 3 different processes, each using different materials:

The CNC quote used aluminum - a strong material with great thermal properties. To be clear, I never intended to use aluminum, Xeometry just didn't let me select a different material. The casting quote used a PC-like material (not sure` what "polycarbonate-like" actually means, but W.E.). And the 3D printing quote defaulted to ABS plastic (not ideal for outdoor use but an alright material) - the quote was so outrageously high I didn't bother switching to my intended material. The first thing I had to come to terms with was that outsourcing was expensive, regardless of the process. Aluminum was nice, but having 2 copies of the case would have cost at least ~$1800. Casting usually is a good idea because then you get a reusable cast that normally leads to shorter lead times and lower costs, but getting 2 copies would have cost at least ~$6000 - I could buy a soccer team with that much money! Last, 3D printing was almost affordable, but with a cost ~$1000 I thought it might be in my best interest to buy my own 3D printer. Coming to this realization is how I arrive at this post!

Here's a table summarizing my search:

Something I expect readers to wonder is, "where did these printers come from, there's so many of them, how did these make the cut?" Here's my methodology. First, I picked brands I knew - that's how Prusa made it onto the list. Then, I consulted Copilot for popular 3D printer brands - that's how the rest of the brands it onto the list.

Now, I have justified how the brands made it onto the list. Next, I picked a brand I recognized but knew nothing about to begin exploring. I did this because, in my experience shopping I've recognized that all normally have product "tiers" and the product in a specific "tier" is normally priced competitively compared to another product in the equivalent "tier" from another brand. So, if I explored the Bambu brand I could expect to find 3D printers at different price points, then I could use those price points to find the equivalent "tiered" printer from anothe brand. By identifying the price point of a "tier" from a single brand I save myself from analysing all of the printers for all of the brands. I chose the Bambu brand becuase they are the brand that my local public library has available for rent.

Exploring Bambu was great! They had many options that I enjoyed looking at. If I didn't know what I was looking for I'd've thot they were all great printers. Fortunately, that was not the case - I knew what I was looking for. I knew what I was looking for because I knew (or thot I knew) I wanted to print PC-GF30 (polycarbonate-fiberglass 70/30). PC-GF30 has a few unique properties that make it an advanced material to print. Specifically, the roll of filament sold by Forward-am requires the printer nozzle temperature to be between 280 and 330 degrees celsius, the bed temperature to be between 80 and 100 degrees celsius, and (altho not explicitly stated in the printing guidelines) a hardened nozzle (normally hardened steel) + filtered enclosure. Thanks to these requirements/guidelines I could eliminate every entry level printer because entry level printers cannot achieve the required temperatures. This left me with only mid-tier printers that were characterized by a price point near $1000.

The Bambu model that achived these requirements was the H2S ($1400), the Flashforge model was the Creator 5 Pro ($850), the Creality model was the K2 Plus ($1200), and the Prusa model was the One+ (Gen 2) (note, these models don't all achieve the guidelines but they are in the same price tier as the Bambu model). All 4 models achieved the hotbed temperature (all achieved 120C). Only 2 models achieved the required nozzle temperature (the Bambu and the Creality), the Prusa model capped at 290C (they have a $200 hotend upgrade that does achieve this but is currently sold out) and the Flashforge model caps at 320C which is 10C less than the max suggested temperature (which led me to believe this might be unreliable in practice). Only 2 printers came with a hardened steel nozzle by default (I could not find what the Flashforge nozzle was made of) and 3 of the 4 brands sold compatible hardened steel nozzles (again, could not verify the material of the falshforge nozzles) - therefore, all brands except for the Flashforge modle could still be considered. All brands provided 0.6 and 0.8mm nozzles. The enclosure temperature is not specified by the guidelines, but I know that having a higher enclosure temperature helps with printing this material, thankfully all printers have an actively heated chamber.

Next, I could not verify what type of gears the Flashforge brand used in its printers, but I could verify that all other printers have hardened steel extruder gears (the table shows that I could not verify this for the Prusa printer but I did verify this after). The prusa has the smalles printbed of the printers that still serve as viable candidates, the Bambu and K2 have much larger bed sizes tho and are much taller (thus, have greater volume). All remaining printers by defualt support 1.75mm diameter filament - this is only useful when buying filaments, it means I can only buy filaments of this size. The layer height was only available for 2 printer, but this is not a very useful characteristic because the print layer height changes based on the nozzle being used. All printers have support for some sort of advanced filtration system which is nice because PC-GF30, as well as other engineered filaments, release toxic fumes. Here is a stat that stood out - only the H2S supports having a filament drying attached to it! Unfortunately, the most powerful dryer that the H2S supports can dry up to 85C, but PC-GF30 needs to be dried at 90C (maybe, might depend on brand). It may be possible to dry it anyway but it would take several hours longer to dry at such a low temperature. PC-GF30 is highly hygroscopic (absorbs moisture) so drying it is almost a necessity. If it is not properly dried and store before use, it will lead to stringy extrusion, blobs forming at the nozzle tip, and other undesired things.

Technically speaking, no printer supports PC-GF30 - they all need something extra. This meant that no matter which printer I chose, I'd need something extra - the prusa needs a hotend upgrade and a drying, the other Bambu potentially needs a stronger dryer, the K2 pro needs an external dryer. I was in a loss-loss situation. The last important fact is the rated power supply. Now, I don't know much about power supplies but seeing >1000W on the data sheets for the printers threw me off pretty hard. The rated power supply is actually what made me side with the Prusa over the others. The Prusa is rated at 240W, much less than the others. I thot, if these other guys are consuming this much power there has to be something wrong. That, in addition to the way the Prusa business is ran (warehouses stocked with replacement parts for decades) also made me lean towards prusa. That's why I chose to buy the Prusa. Now, I still had an issue, I need to print the enclosure and I can't get a hotend upgrade - what do I do!? The answer - use a different material. Instead of printing with PC-GF30 (a highly advanced filament to print) I've decided to print with ASA - a still highly suitable plastic for my application that produces less toxic fumes, is cheaper, and is easier to print (tho, it is stil significantly more complex to print than PLA). That's why I decided to get the Prusa printer!

Opinion

Initial Opinion on Mechanical Drawings

I was working on my thesis project one morning when I suddenly realized, "my raspberry Pi won't last 5 minutes outside in the winter". CAD, specifically OnShape & SOLIDWORKS, has been on my mind recently (a lot) so the first thing that made sense was to design my own enclosure! (see below)

This design was OK200 but it wasn't done. I have plenty of changes in mind to make to it to have it be ready for everything the outside world will put it thru. Sadly, it took me about 6 hours to develop this initial design - far long than it would have taken me in my CAD prime during my first year @North Central College - and, quite frankly, the thot of making more changes to this design irritated me. I thot, "what if I ask Copilot to help me?"

The first thing I ask Copilot was:

As usual, the response was optimistic. The initial response led me to believe that I could directly upload .STL or .STP files to Copilot and I could ask it for recommendations. Unfortunately, 2 inquiries later I learned that this was not the case:

So.. Copilot said I could upload screen-grabs!

Now I had a different problem, previously I've had a CAD prime but I've never had a mechanical drawings prime - I've never made a mechanical drawing. Thanks to my time working in CAD I'd at least noticed the "drawings" button in the software. After a another 5 or so hours of clicking and playing around with the feature I came up with this:

It's definitely not finished, but, I had other tasks to attend to so that's where I left it before staring this blog post. I thot, "what did I just do, what did I do this for, and what is this good for?" My answer - I just build documentation to help me communicate with an AI. So, referencing the topic of this post, my initial opinion on mechanical drawings is that they are good for communicating. When I used the feature it gave me the option to import a drawing from a CAD design. It also let me import measurements based on the CAD drawing. Making this drawing felt like it was naturally supposed to come after the CAD design! This makes me feel like it's alright to designs something that I haven't yet been able to describe on paper, because I can describe it on paper after it's been designed. If this is the case, then mechanical drawings are good for communicating your idea after you've designed it. It displays measurements, different orientations, materials, titles, and other important information - all on one document! If I had a team, I could give them the drawing and they could view it on their cellphone and understand what my enclosure design looks like, what it is made of, and how it works.

In conclusion, thru this work I've convinced myself that mechanical drawings, often times called engineering drawings, are useful for communicating designs with team members & AIs. This opinion was formed after my first time using CAD software to make a drawing. This, however, is not my first time making drawings. I used to frequently make mechanical drawings while an undergrad in engineering, but I would make these drawings using pen and paper!! Then, when I learned CAD, I would take my detailed hand drawn artifacts and quickly layout appropriatly-shaped planes which I could cut and extrude to create my designs. Now, after a few years of being out of practice I've developed a design before the drawing. In the past, my drawings were useful for communicating to myself what I want to create. Now, they have become useful for communicating to others what I have created. A lesson learned is that I should` learn to create drawings in CAD software :_)

Incident

Investigating Code Generation Hallucinations

I made a cool web app recently. It's useful for tracking workouts. Check it out @https://diego-esquivel.github.io/Workout-Tracker/.

Here's the prompt I used to generate the code. It's basic, it's not all the "correct", there are no UML charts, and I made it up in 30 seconds - BUT it got the job done (a testatment to the power of LLMs).

Like most LLMs, the initial response was both confident and voberse:

Sadly, it was also filled with simple syntax errors. Notably, it didn't add the "link"j, "script", or "anchor" tag around the the appropriate stylesheet, app.js, or hyperlinks. This wasn't a very complicated app so I was highly dissapointed. In the LLMs defense, my prompt was not very good.

Next, I wanted to see if it could correct itself... it could talk about its mistakes but it could not actually correct them...

I also tried asking it for help but it gave me bad advice.

If you read my first post you'll notice this is an example of it saying 1 thing that is exaclty what I was talking about but everything else is totally wrong. The only sensible thing I could think to do now was to start a new agent!!

Instead of manually copy and pasting the correction, I asked it to summarize the results.

This resulted in good code (::).

For whatever reason, the first agent was only good at generating initial code and could not provide syntax help. The second agent could provide syntax help. This could be due to my poor prompt BUT investigating that is for another post.

I also could try sounding more professional, but I'm the whole thing process took 15 minutes so, not exactly worth my time.

Finally, If I try to use AI for a more sophisticated project I will be sure to provide it with a more advanced prompt. In the past, this has yielded mildly pleasant results. The art of prompt engineering feels promising.

Incident

Investigating Unrecognized Network Devices

I was snooping in my router's settings when I noticed a couple devices I couldn't recall adding to my network.

Here's what my router's management page looks like.

I was snooping around it when I noticed 2 devices I didn't recognize - "192.168.1.242 / Linux" and "192.168.1.111 / espressif"

I didn't want to risk the Chinese government being in my home network so I launched an investigation. The first step was to see if they had any open ports. The espressif device had port 80 open. The Linux device has port 6668 open and 27 filterd ports.

Next, I wanted to see if they responded to a basic CURL request. Espressif did, but Linux timed-out.

Here's what the espressif device returned on a web-browser.

This information was nice but I still had no clue what devices they were. A quick Google search revealed that the MAC addresses for those devices were registered to ESPressif Inc - a huge China-based MCU designer - and the Linux devices was registered to TUYA (another big IoT device company). This was insightful, but I still didn't know what devices they were.

I blocked their MAC addresses in my router's configuration and nothing stopped working at home, so it couldn't have been anything I set-up. I flicked on and off all of my lightbulbs and smart speakers but they wouldn't dissapear form the network.

It wasn't until after I disconnected my siblings water kettle that the espressif device disconnected from the network - myster half solved! It turned out the water kettle was a smart device added to the network without my knowledge.

I still didn't know what the Linux device was. It's signal strength was -85mdB - super weak - and besides for it's MAC and IP address I didn't know anything else about it. I wanted to figure out how far from the router it could be.

Finally, I connected my cellphone to the router and walked away from the router. I ended up going outside, into the backyard, past the patio, and sat in a chair next to a firepit - almost off my proprety. My signal strength read "-81mdB"! Even though I was almost off my property I still had a stronger signal than the Linux devices. Going into my neighbors yard/house was off the table so, with nothing else to do, I settled for simply blocking it's MAC address and hiding my network SSID. As long as I regularly monitor for the reappearance of this device my network should remain uncompromised.

Incident

Benchmarking Hallucination Detection

Have you ever read a response from your AI and thot "this is totally not what I was talking about.... but wait, this bit (& this bit only) is exactly what I was talking about".

Read the heading. This is my first post. The topic of this heading is the most common hallucination I've encountered. When this happens it is important to remember that AI is ONLY a tool for you (the sophisticated one) to leverage. This is your opportunity to grab the AI by the handle-bars and steer it in the appropriate direction. Start a brand new chat on a newly grounded (& more evolved) topic.