qlyoung's wiki
Page Tools
User Tools
    Trace:

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
bird_bar [2024/07/03 19:16] – change ai tag to machine learning qlyoungbird_bar [2026/02/08 00:58] (current) – update history section qlyoung
Line 1: Line 1:
 ====== bird bar ====== ====== bird bar ======
  
-{{ :feeder-with-chickadee.jpg?200|Bird feeder with Carolina chickadee perched on it}}+Bird bar is a bird feeder with a camera on it. Each frame from the camera's video feed is run through a classifier model to recognize any birds in the picture. Each bird is labeled with its species above its head and the resulting video is live-streamed to the internet. Species data is collected in a time series database.
  
-At the start of 2021 I received a window-mount bird feeder as a secret santa gift (thank you!). As someone who loves birds I was excited to put it up and get a close up view of some of the birds that inhabit the woods around where I live. It’s a great little feeder and within around 3 days I had birds showing up regularly.+Live stream: https://twitch.tv/thebirdbar
  
-Shortly after installing the feeder I had the idea to mount a camera pointing at it and stream it to Twitch, so that I could watch the birds while I was at my computer. While watching I found myself wondering about a few of the species I saw, and looking up pictures trying to identify them. Then it hit me - this is a textbook computer vision problem. I could build something that used realtime computer vision to identify birds as they appeared on camera.+Stats: https://birdcam.qlyoung.net/
  
-Fast forward a few years and this has bloomed into a pretty large project. I run two feeders, one for all birds and one for hummingbirds. Both of them are livestreamed to Twitch. It's definitely the most popular project I've ever made; my friends think it's cool, and at $dayjob my current manager brought it up during my interview since he'd seen it on my website.+{{gallery>:birdbar:screenshots}}
  
-===== The Feeder =====+===== History ===== 
 + 
 +At the start of 2021 I received a window-mount bird feeder as a secret santa gift. As a bird lover I was excited to put it up and get a close up view of some of the birds that inhabited the woods around where I lived. Within around 3 days I had birds showing up regularly. 
 + 
 +With the floor plan of my apartment at the time, the only sensible place to put the feeder was on the kitchen window; there was a screened porch on my bedroom window, or I would have put it there. Since my work desk was in my bedroom, this meant that I couldn't watch it while I worked. If that had been an option, the rest of the project may never have materialized. 
 + 
 +Shortly after installing the feeder I had the idea to mount a camera pointing at it and stream it to Twitch, so that I could watch the birds while I was at my computer in another room. While watching I found myself wondering about a few of the species I saw and looking up pictures trying to identify them. Then it hit me - this is a textbook computer vision problem. I could build something that used realtime computer vision to identify birds as they appeared on camera. 
 + 
 +Fast forward a few years and this has bloomed into a pretty large project, with multiple upgrades to both the hardware, software and feeder setup. It's definitely the most popular project I've made; my friends think it's cool. It's also served as a good test bed to keep up to date on advances in machine learning and accelerated computing. 
 + 
 +===== Feeder =====
  
 This section covers the evolution of the feeder construction & installation details. This section covers the evolution of the feeder construction & installation details.
  
-With the floor plan of my apartment, the only sensible place to put the feeder was on the kitchen window; there’s a screened porch on my bedroom window, or I would have put it there. This meant that I couldn't watch it while I worked since my desk is in another room. If that had been an option, the rest of the project may never have materialized.+==== v1 ====
  
 Initially the feeder was mounted 'stock'. The camera was an old webcam I had lying around. Since it's mounted outside it needed to be weatherproofed. I did that with plastic wrap. Subsequent attempts greatly improved the design. Initially the feeder was mounted 'stock'. The camera was an old webcam I had lying around. Since it's mounted outside it needed to be weatherproofed. I did that with plastic wrap. Subsequent attempts greatly improved the design.
  
-{{:feeder-with-camera.jpg?400|Bird feeder showing webcam pointed at it}} {{:feeder-with-camera-outside.jpg?200|Picture of webcam attached to the side of my apartment building}} {{:webcam-condom.jpg?200|Picture of the webcam completely wrapped in plastic wrap secured by orange duct tape sitting on my window sill}}+{{birdbar:feeder:feeder-with-camera.jpg?400|Bird feeder showing webcam pointed at it}} {{:feeder-with-camera-outside.jpg?200|Picture of webcam attached to the side of my apartment building}} {{:webcam-condom.jpg?200|Picture of the webcam completely wrapped in plastic wrap secured by orange duct tape sitting on my window sill}}
  
 {{ :webcam-tupperware-lid.jpg?200|Picture of a tupperware lid taped over the camera as a sort of primitive rain shield}} {{ :webcam-tupperware-lid.jpg?200|Picture of a tupperware lid taped over the camera as a sort of primitive rain shield}}
Line 29: Line 39:
 ===== Bird Identification ===== ===== Bird Identification =====
  
-{{ :me-with-phone-yolo-detection.png?200|Screen capture of webcam feed after applying YOLOv5's out-of-box 'small' model to a scene of me holding up my cell phone. Picture is heavily blurred}}+Birds arriving at the feeder are identified using [[https://github.com/ultralytics/yolov5|YOLOv5]] fine tuned on [[https://dl.allaboutbirds.org/nabirds|NABirds]]. 
 +==== Background ====
  
 I’d read about [[https://pjreddie.com/darknet/yolo/|YOLO]] some years before and began to reacquaint myself. It’s come quite far and seems to be more or less the state of the art for realtime computer vision object detection and classification. I downloaded the latest version ([[https://github.com/ultralytics/yolov5|YOLOv5]] at time of writing) and ran the webcam demo. It ran well over 30fps with good accuracy on my RTX3080, correctly picking out myself as “person”, my phone as “cell phone”, and my light switch as “clock”. I’d read about [[https://pjreddie.com/darknet/yolo/|YOLO]] some years before and began to reacquaint myself. It’s come quite far and seems to be more or less the state of the art for realtime computer vision object detection and classification. I downloaded the latest version ([[https://github.com/ultralytics/yolov5|YOLOv5]] at time of writing) and ran the webcam demo. It ran well over 30fps with good accuracy on my RTX3080, correctly picking out myself as “person”, my phone as “cell phone”, and my light switch as “clock”.
 +
 +{{:me-with-phone-yolo-detection.png?400|Screen capture of webcam feed after applying YOLOv5's out-of-box 'small' model to a scene of me holding up my cell phone. Picture is heavily blurred}}
  
 Out of the box YOLOv5 is trained on COCO, which is a dataset of _co_mmon objects in _co_ntext. This dataset is able to identify a picture of a Carolina chickadee as “bird”. Tufted titmice are also identified as “bird”. All birds are “bird” to COCO (at least the ones I tried). Out of the box YOLOv5 is trained on COCO, which is a dataset of _co_mmon objects in _co_ntext. This dataset is able to identify a picture of a Carolina chickadee as “bird”. Tufted titmice are also identified as “bird”. All birds are “bird” to COCO (at least the ones I tried).
  
-{{:chickadee-2.jpg?200|Image of chickadee with a poorly sized bounding box drawn around it with the label "bird" and a confidence rating of 0.31}}+{{:chickadee-2.jpg?400|Image of chickadee with a poorly sized bounding box drawn around it with the label "bird" and a confidence rating of 0.31}}
  
 Pretty good, but not exactly what I was going for. YOLO needed to be trained to recognize specific bird species. Pretty good, but not exactly what I was going for. YOLO needed to be trained to recognize specific bird species.
  
-===== Dataset =====+==== Dataset ====
  
 A quick Google search for “north american birds dataset” yielded probably the most convenient dataset I could possibly have asked for. Behold, [[https://dl.allaboutbirds.org/nabirds|NABirds]]! A quick Google search for “north american birds dataset” yielded probably the most convenient dataset I could possibly have asked for. Behold, [[https://dl.allaboutbirds.org/nabirds|NABirds]]!
Line 210: Line 223:
 In the case of sexually dimorphic species that also have appropriate training examples, such as house finches, it’s even capable of distinguishing the sex. In the case of sexually dimorphic species that also have appropriate training examples, such as house finches, it’s even capable of distinguishing the sex.
  
-{{ :finches.mp4 |}}+{{ birdbar:screenshots:finches.mp4 |}}
  
 In a few cases, such as the nuthatch and the pine warbler, the model taught me something I did not know before. Reflecting on that, I think that makes this one of my favorite projects. Building a system that teaches you new things is cool. In a few cases, such as the nuthatch and the pine warbler, the model taught me something I did not know before. Reflecting on that, I think that makes this one of my favorite projects. Building a system that teaches you new things is cool.
Line 232: Line 245:
 Then I thought it would be cool to show these graphs on the livestream. It turns out Grafana supports embedding individual graphs, and since OBS supports rendering browser views it was easy to get those set up. Then I thought it would be cool to show these graphs on the livestream. It turns out Grafana supports embedding individual graphs, and since OBS supports rendering browser views it was easy to get those set up.
  
-{{:stats-display.png?400|screenshot from the stream with a bar graph showing species counts}}+{{birdbar:screenshots:stats-display.png?400|screenshot from the stream with a bar graph showing species counts}}
  
 I left these up for a while, but ultimately I felt they were taking up too much space in the stream so I took them down. I left these up for a while, but ultimately I felt they were taking up too much space in the stream so I took them down.
Line 245: Line 258:
   * Retrain with background images to reduce false positives   * Retrain with background images to reduce false positives
  
-{{:bluebird-and-chickadee.png?400|Eastern bluebird and Carolina chickadee feed together}}+{{birdbar:screenshots:bluebird-and-chickadee.png?400|Eastern bluebird and Carolina chickadee feed together}}
  
  
Panorama theme by desbest
bird_bar.1720034178.txt.gz · Last modified: by qlyoung · Currently locked by: qlyoung
CC Attribution-Noncommercial-Share Alike 4.0 International Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 4.0 International