Wordpress, MySQL Server, WooCommerce, PHP
We were approached by “Client A” to create a project to create an automated csv and zip file import process for product data. This import process would integrate as a WordPress Plugin into an existing WooCommerce store. This import procedure could be initiated manually by uploading a csv file or by scanning a specific folder for csv files. The product data would be imported via CSV files which would be automatically uploaded to the clients local server via FTP transfer.
In the process of implementing the solution we quickly realized that the import procedure would be severely bottlenecked by processing the files using PHP to read each row in the CSV file. Some files would have over 20,000 rows which could take 4-5 minutes to perform reading and importing often with even slower response times. In order to address this issue we utilized a large database import of the csv data into a custom MySQL database structure and performed our data merge by utilized stored procedures to insert the product data in large batches. This decreaed import times to below 1 minute even in the case of very large files.
In some cases csv and zip files would fail during FTP transfer leaving a partial corrupted file in the FTP folder. In order to address this we implemented a try catch style of scanning the directory. We would attempt to import the csv or extract the zip file. If the import was successful we would temporarily store the successful imports in a last import directory. If the import or extraction met with issues we placed the files in questions into an failed to import file folder and emailed a message to the site administrator so they could review the files themselves or reattempt the ftp transfer from their data provider.Final Completion: Upon final delivery of our solution we had a wordpress plugin that successfully import tens of thousands of products into a WooCommerce store allowing the site owner to save hours of time creating products manually. In addition updates to pricing, product data, and images became an automated process so the site owner could focus on store management and sales rather than data management.