Introduction to the Capstone Project
This capstone project introduces a web-based application that uses artificial intelligence to classify skin lesions and detect monkeypox. The primary aim was to create an accessible educational tool that provides preliminary insights into potential skin conditions, without acting as a substitute for professional diagnosis. The application is designed to deliver predictions in under two seconds without storing any user data, ensuring both privacy and speed.
The app operates on a minimalist stack, combining Flask for the backend, TensorFlow Lite for machine learning inference, and Tailwind CSS for styling. This simplicity ensures that it remains lightweight and runs efficiently even on free-tier deployment servers. The project emphasizes early detection of conditions like melanoma, which could potentially save lives by encouraging users to consult dermatologists sooner.
Technical Architecture and Tools
The technical stack was deliberately kept simple to streamline development and deployment. The backend is built using Flask, a lightweight Python web framework, while the frontend leverages Tailwind CSS and Alpine.js for a clean and interactive user experience. Machine learning inference is powered by TensorFlow Lite, a framework optimized for lightweight and fast model execution. These components work together to provide a seamless user experience.
The app also integrates Groq's Llama 4 Scout to generate plain-language explanations for predictions. This multimodal AI model can analyze uploaded images and transform raw CNN outputs into easily understandable medical explanations. This feature enhances the app's utility by making complex medical data more accessible to users who lack specialized knowledge.
Model Training and Challenges
The primary machine learning model is a seven-class skin lesion classifier trained on the HAM10000 dataset, which contains over 10,000 dermatoscopic images. The training process relied on a MobileNetV2-based CNN, chosen for its balance between performance and computational efficiency. The final model size of 27 MB allows it to be loaded instantly, even on resource-constrained servers.
A significant challenge was handling dataset imbalance, as melanocytic nevi dominated the dataset with 67% of images. To address this, the model was fine-tuned to improve its generalization capabilities. In addition to the primary classifier, a separate binary model for monkeypox detection was developed. This model achieved an accuracy of 95%, benefiting from the distinct visual features of monkeypox lesions.
Data Privacy and User Experience
A standout feature of the application is its commitment to data privacy. Uploaded images are processed entirely in memory, with no data stored on disk or in a database. This approach ensures that user data is not retained, reducing the risk of data breaches. Once processed, the image data is discarded, and garbage collection handles any residual memory usage.
From a user experience perspective, the app prioritizes speed and simplicity. The TensorFlow Lite models are loaded into memory during server startup, enabling near-instantaneous predictions after the initial cold start. This design ensures that users receive quick feedback without compromising the accuracy of results.
Educational Value and Limitations
The app is explicitly marketed as an educational tool rather than a diagnostic device. Its primary purpose is to provide users with an informed starting point for seeking professional medical advice. The inclusion of Groq's API-generated explanations adds significant value by guiding users through potential conditions, visible signs, and recommended actions.
However, the tool has its limitations. The skin lesion classifier achieves an overall accuracy of 71.64%, which is insufficient for clinical-grade applications. Despite its high accuracy for binary monkeypox detection, the app is not a substitute for a dermatologist's expertise. Users are cautioned to rely on the tool for educational purposes only and to consult healthcare professionals for proper diagnosis and treatment.