AI-Powered Parts Classification Workflow for Large CSV/Excel Data

First, this task cannot be done in one shot. You need to break it down into simpler tasks and let AI help you complete them semi-manually. Difficulties: 1. Need to read Excel data 2. Need AI to classify each record 3. Need to write results back to Excel 4. Cannot process 20,000 records at once; need to split into multiple batches and then merge Here's how I would approach it: 1. Export Excel to CSV, keeping only fields needed for classification, but two fields are mandatory: Part ID and Part Description. Large language models handle text well and it's cost-effective; pure text should suffice. Multimodal would be more complex. 2. Split the large CSV into multiple smaller CSV files using a program. 20,000 records won't fit in the context window. Test how many records work best per batch (e.g., 100, 50, 10). 3. Write a prompt that generates classifications based on input part IDs and descriptions. This is the core task: a) Predefine categories for the LLM to choose from b) Define output data structure (JSON, XML, YAML, or CSV) for easy parsing c) Write the prompt to output the desired structure Example prompt for GPT-4o: I have an Excel with 20,000 rows of fragmented part descriptions. I want AI to classify each row. Please generate: - 5 detailed part descriptions, about 1 paragraph each - A complete set of part categories, about 10 items - A prompt that takes part IDs and descriptions as input and returns the corresponding category in JSON format. Input example: [{"id": "123", "description": "XXXXX"}, {"id": "345", "description": "YYYY"}]. Output example: {"123": "cat1", "345": "cat2"} 4. Write a program to call the LLM API: a) Read one small CSV file at a time b) Parse CSV to get part list c) Call LLM API with the prompt and part list d) Parse API response to get ID-category mapping e) Save as new CSV with categories added f) Repeat until all files are processed 5. Merge all categorized CSV files into one large CSV and import back into Excel. The key is to break complex problems into smaller ones and use AI for each step. When you hit issues, ask the LLM for help.