This work addresses the challenge of creating interactive learning environments by using Multimodal Large Language Models (MLLMs) to automatically answer student questions derived from online lecture videos. We introduce the EduVidQA dataset containing 5,252 question-answer pairs curated from 296 computer science lecture videos. We built a data generation pipeline combining manual annotation with LLM-assisted question synthesis (Gemini 1.5 Flash), producing 4,300+ Q&A pairs. We benchmark six state-of-the-art multimodal LLMs (Video-LLaMA, mPLUG-Owl, etc.) across zero-shot, few-shot, and fine-tuned settings, evaluating them using both text-based metrics (BLEU, ROUGE, METEOR) and qualitative student feedback.
@inproceedings{ray2025eduvidqa,title={EduVidQA: Generating and Evaluating Long-form Answers to Student Questions based on Lecture Videos},author={Ray, S. and Sharma, Shubham and Aditya, S. and Goyal, P.},booktitle={Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP)},year={2025},}
2024
CompSec
A Hybrid CNN-LSTM Approach for Intelligent Cyber Intrusion Detection System
S.
Bamber, A.
Katkuri*, Shubham
Sharma*, and
1 more author
We develop a deep learning-based Intrusion Detection System using the NSL-KDD dataset. Feature selection is optimized using Recursive Feature Elimination (RFE) with a Decision Tree classifier. The hybrid architecture combines two sets of 1D convolutional layers (64 and 128 filters) with max-pooling and batch normalization for spatial feature extraction, followed by a 100-unit LSTM layer for capturing temporal dependencies. The model achieves 95% accuracy and 0.94 F1-score for binary classification of network traffic as normal or intrusive.
@article{bamber2024hybrid,title={A Hybrid CNN-LSTM Approach for Intelligent Cyber Intrusion Detection System},author={Bamber, S. and Katkuri, A. and Sharma, Shubham and Angurala, M.},journal={Computers \& Security (Elsevier, IF: 6.8, Q1)},year={2024},}
ICDAR
IndicBART alongside Visual Element: Multimodal Summarization in Diverse Indian Languages
D.
Prakash*, Raghvendra
K*, S.
Saha, and
1 more author
In International Conference on Document Analysis and Recognition (ICDAR), 2024
We develop IndicBART, a multimodal summarization framework that integrates text and image data to produce regionally contextualized summaries in Indian languages. We collect a multimodal dataset (Hindi, Tamil, Bengali, Marathi) with paragraphs, summaries, and images from the Large Scale Multi-Lingual Multi-Modal Summarization Dataset (M3LS). We extend the BART model with a visual-aware encoder and achieve a ROUGE-1 score of 0.266 on the Hindi dataset and 44.1% image precision using an image pointer for multi-output prediction.
@inproceedings{prakash2024indicbart,title={IndicBART alongside Visual Element: Multimodal Summarization in Diverse Indian Languages},author={Prakash, D. and K, Raghvendra and Saha, S. and Sharma, Shubham},booktitle={International Conference on Document Analysis and Recognition (ICDAR)},year={2024},}
RSR
Forecasting of Crop Yield Using Various Machine Learning Approaches: A Comparison
Shubham
Sharma, G. K.
Walia*, K.
Singh*, and
5 more authors
We perform a comparative evaluation of various machine learning and deep learning approaches for crop yield forecasting. The study evaluates Random Forest, Extra Trees, Artificial Neural Networks, SVR, and Decision Tree Regression on agricultural datasets incorporating climatic parameters (rainfall, temperature), soil composition, pesticide usage, and historical yield data. Models are validated using RMSE, MAE, and R-squared scores with cross-validation, providing practitioners with insights into selecting the most robust models for precision farming and resource allocation.
@article{sharma2024forecasting,title={Forecasting of Crop Yield Using Various Machine Learning Approaches: A Comparison},author={Sharma, Shubham and Walia, G. K. and Singh, K. and Batra, V. and Sekhon, A. K. and Kumar, A. and Rawal, K. and Ghai, D.},journal={Rural Sustainability Research},volume={52},number={347},year={2024},}
2023
ECIR
Knowing What and How: A Multi-modal Aspect-Based Framework for Complaint Detection
A.
Singh*, V.
Gangwar*, Shubham
Sharma, and
1 more author
In European Conference on Information Retrieval (ECIR), 2023
We introduce the task of Aspect-Based Complaint Detection (ABCD) and propose a deep-learning-based multi-modal framework that processes both text and visual information from Amazon product reviews. The framework operates in two stages: (1) Aspect Prediction to identify targeted product features, and (2) Complaint Categorization to classify complaint severity. We collect a multimodal dataset from Amazon, extract textual embeddings using BERT and visual representations using pre-trained CNNs (VGG16, ResNet), and develop a multimodal interaction model to learn the relationship between text, image, and product aspects for identifying complaint eligibility.
@inproceedings{singh2023knowing,title={Knowing What and How: A Multi-modal Aspect-Based Framework for Complaint Detection},author={Singh, A. and Gangwar, V. and Sharma, Shubham and Saha, S.},booktitle={European Conference on Information Retrieval (ECIR)},year={2023},}
CPAMCS
Pneumonia Detection in Chest X-Rays using XGBoost based Meta-learner with Deep Feature Extractors
S. Mukherjee*, Shubham Sharma*, D. Kaplun, and R. Sarkar
We develop an effective pneumonia detection system by combining advanced image analysis with ensemble learning. We compile and preprocess a Mendeley dataset with three classes (Covid-19, Pneumonia, Normal) and apply data augmentation to address overfitting. Image features are extracted using Vision Transformer (ViT) and ResNet50 models, then combined into an ensemble. XGBoost is used as a meta-learner classifier on the concatenated features, achieving 96.19% accuracy on the pneumonia detection task.
@inproceedings{sharma2023pneumonia,title={Pneumonia Detection in Chest X-Rays using XGBoost based Meta-learner with Deep Feature Extractors},author={Mukherjee, Sutirtha and Sharma, Shubham and Kaplun, Dmitrii and Sarkar, Ram},booktitle={CPAMCS 2023, Springer},year={2023},}
