We occasionally sell large items that are shipped directly from the supplier to the customer. Since the large item is never in my warehouse, I'm thinking that transaction should not affect my inventory asset account - am I right? If so, what is the GAAP for handling this transaction?

Accounting for inventory sold directly from supplier
Answers
Why not regard it as in + out inventory from a warehouse location that never has a balance? That way you use the standard inventory functionality of the ERP system; it's simply a sales order that spawns a PO on a vendor who ships on your behalf.
As soon as you get the vendor shipment notification, you:
1-receive the inventory into your "drop ship" warehouse- match to PO
2-then immediately "ship confirm" it and generate the invoice/recognize the revenue.
3-result: warehouse inventory balance is zero; COGS matches Sales. You are using standard order management processes, except that you are using a virtual location for the drop ship inventory.
This way your sales forecasting is holistic, and so is your supply chain planning.
Rhetorical points...
1. Why are you going to pay your supplier if you did not took possession/ownership of the goods?
2. How are you going to sell something you do not own?
Inventory (or ownership) is not dependent on "location".
No matter how you do it, it will (or should) hit/affect your inventory account. (See Len's answers). It will first hit from the AP side.
We call the transactions you describe "drop ships" and we have modified our ERP system to treat them differently than regular inventory orders. However, at the end of the day, the routing Len describes is what our system does. The customization, just simplifies the process to eliminate the need for a third department to "receive" the inventory. One department places the drop ship order with the vendor. Either that same department or customer service will record the drop ship sales order. When A/P gets the invoice and tries to match it in the system, the system matches the PO and vendor invoice and automatically creates the receiver and invoices the customer because of the designation as a "drop ship." Your actual receiver - the customer - will let you know if they did not get the product you billed them. The sales department will let A/P know if the drop ship order is aging (in case it was shipped, but the invoice lost) because they want credit for the sale. The system will kick out if the vendor bills for more than what was ordered.
Deanne Miller's explanation is spot on. In the computer hardware distribution business "drop ships" have been the normal course of business for decades, more business models are using this technique to reduce inventory and speed delivery. With revenue recognition being what it is FOB terms and proof of delivery will keep you on the auditor's good side.