Campus Intelligent Bus System

Based on the YOLO deep learning framework, an intelligent unmanned campus bus.

ROBOTICS CONTROL LOOP

Perception, decisions, chassis, and monitoring in one loop

LiDAR, depth imagery, and recognition describe the campus while ROS turns perception into avoidance, stopping, and motion.

13sign classes

300images per class

>80%test accuracy

01 SENSE LiDAR + depth

02 PERCEIVE YOLOv5n

03 DECIDE ROS nodes

04 ACT Mecanum base

05 MONITOR Web console

LiDAR

YOLOv5n

Jetson Nano

Flask API

COREROS

DESIGN DECISION

Edge compute and remote UI are separated

Jetson Nano performs recognition and control. Flask exposes status and commands, while React handles live monitoring, stops, and manual takeover.

ROSJetson NanoTensorRTReactFlask

CAPABILITIES

Campus tasks

01Obstacle avoidance

02Sign detection

03Precision stop

04Remote control

Introduction to the Vehicle

Vehicle Introduction

Campus Simulation Environment Introduction

Model Results

For the target recognition requirements, we adopted the smallest Yolov5 nano model in the Yolov5 framework as the training model to ensure the highest recognition efficiency. To achieve precise recognition, we constructed a dataset containing 13 categories for model training, with 300 images collected for each category. Since the generalization ability requirement of the model in this project is not particularly high, during the training process, we used a higher number of training epochs on a larger training set to train the model as fully as possible, resulting in an accuracy rate of over 80% on the test set. By recognizing road signs, the current station is identified, and using the distance returned by the depth camera to the road sign, the campus bus can stop at an appropriate distance to allow passengers to get on and off.

Frontend Introduction

This project is the frontend part of an intelligent unmanned campus bus based on the YOLO deep learning framework, with main functions including:

Real-time acquisition of campus bus camera information
Real-time acquisition of station information
Real-time monitoring of the campus bus location
Real-time monitoring of the campus bus passenger load
Remote manual control of the campus bus

Frontend Effect

Project Features

Development Notes: React NextJS App Router Real-time API Fetching

On the campus bus, a Flask framework is used to build an API server, which obtains campus bus data and sends control signals through the API.

The frontend of this project is mainly developed using React under the NextJS framework. While implementing basic functions, when faced with the need to fetch data in real-time, I did not use other third-party libraries to achieve API fetching. Instead, I implemented data-fetching based on the NextJS official documentation.

Finally, after various attempts and combinations, I came up with a reasonable solution: calling getLocationInformation() inside useEffect, and adding next: {revalidate: 0} in the code that fetches the backend API. The code is as follows:

// Code to fetch backend API
export async function GET(request: NextRequest) {
    const res = await fetch(URL + '/api/information', {
        headers: {
            'Content-Type': 'application/json',
        },
        next: {revalidate: 0} // Revalidate data after 0 seconds
    })
    const data = await res.json()
    // return Response.json({ data })
    return NextResponse.json({ data })
}

// Code to call local API
useEffect(() => {
        getLocationInformation();
    });

async function getLocationInformation() {
    const res = await fetch('/api/getLocationInformation');
    if (!res.ok) {
        setMessage("Connect fail!");
        return;
    }
    const data = await res.json();
    setLocationInformation(data.data);
    setRootLocationInformation(data.data);
    for (var _ in data.data) {
        if (data.data[_]["is_arrived"]) setIsNextLocation(LocationInformationLinkedList[_ as keyof TypeLocationInformationLinkedList])
    }
    console.log(locationInformation)
}

My personal understanding is that next: {revalidate: 0} tells NextJS to revalidate the data after 0 seconds when fetching data, which enables real-time data fetching. Additionally, due to the use of the useEffect function, the backend fetches new API data each time, allowing the page to update in real-time. This can be considered a reasonable solution.