We deploy convolutional neural networks to segment endoscopic images to identify areas of interest that can be verified by endoscopists. Currently we are exploring the impact of using both white light and narrow band imaging settings to augment identification of polypoid areas in endoscopic images. This approach can also be used to create embeddings that can generate features for multimodal prediction.