Vision Quest Capstone Project

Project Basics

  • Control a Denso 6 axis robot with LabVIEW

  • Use NI-1772C smart camera with Vision Builder AI find objects and communicate with the Denso control program

  • Create user interface to create and execute vision/robotics programs

  • Use TCP/IP for communication between all key hardware components

  • Use virtualization technology (VMware) for streamlined backups to ensure the project will work in the future

Overview

This capstone project was done spring semester of 2018. The initial idea was to use a camera to find something, and have the robot do something with it. We primarily used equipment that was on hand, a Denso 6 axis robot and a NI smart camera. The NI camera uses Vision Builder AI, which led to the idea of controlling the Denso robot with LabVIEW. The program to communicate and control the Denso robot was built from the ground up using a LabVIEW library from Digimetrix. Our interface has a 'teach' function, allowing a end user to quickly and easily program vision-guided robot movements on their own using drop down menus and VBAI templates. To make the Denso axis control more intuitive, we developed a program for motion control with a Xbox controller. The controller can move each axis individually, as well as control the gripper. It makes our teach interface even easier to use.

In one semester, my partner and I learned a lot about LabVIEW, machine vision, and robotics. It was incredibly challenging, but equally rewarding. I've gained a real appreciation for the field, and hope to work with these technologies in the future.

Network Topology

The pc running the LabVIEW code runs on a virtual machine. All access and programming was done with remote desktop. It was common for us to leave our laptops at school, and use chrome remote desktop to access them from home, then windows remote desktop to access the virtual machine. The NI camera is an embedded system, and runs independently of other hardware.

Vision Configuration

The VBAI program for the candy picker script uses a choose inspection state which will continue to one of three other states when a network variable is updated to meet the transitional condition. After the initial transition, the individual inspections have the same steps with slight differences in the image capture stage to produce the most consistent results. I did not appreciate how important lighting is for machine vision until late in the project, so a polarizing filter was not ordered in time for our use. While it is possible for LabVIEW to more directly control the camera, making the use of VBAI far less necessary, licensing issues precluded us from doing so.


The inspection begins by setting conditional values to zero. An image is captured, and brightness/contrast/colors are adjusted. The inspection then searches for a set pattern, if found a coordinate system is created using the center of the pattern as the origin. The image is displayed on an external monitor, and the inspection status is set to pass to move to the set data state.

In the set data state, network variables are updated to pass the information to the LabVIEW program.