qlyoung's wiki
Page Tools
User Tools

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
bird_bar [2024/02/01 00:15] – [The Feeder] qlyoungbird_bar [2026/02/08 00:58] (current) – update history section qlyoung
Line 1: Line 1:
 ====== bird bar ====== ====== bird bar ======
  
-At the start of 2021 I received a window-mount bird feeder as a secret santa gift (thank you!). As someone who loves birds I was excited to put it up and get a close up view of some of the birds that inhabit the woods around where I live. It’s a great little feeder and within around 3 days I had birds showing up regularly.+Bird bar is a bird feeder with a camera on it. Each frame from the camera's video feed is run through a classifier model to recognize any birds in the picture. Each bird is labeled with its species above its head and the resulting video is live-streamed to the internet. Species data is collected in a time series database.
  
-{{:feeder-with-chickadee.jpg?200|Bird feeder with Carolina chickadee perched on it}}+Live stream: https://twitch.tv/thebirdbar
  
-Shortly after installing the feeder I had the idea to mount a camera pointing at it and stream it to Twitch, so that I could watch the birds while I was at my computer. While watching I found myself wondering about a few of the species I saw, and looking up pictures trying to identify them. Then it hit me - this is a textbook computer vision problem. I could build something that used realtime computer vision to identify birds as they appeared on camera.+Stats: https://birdcam.qlyoung.net/
  
-Fast forward a few years and this has bloomed into a pretty large project. I run two feeders, one for all birds and one for hummingbirds. Both of them are livestreamed to Twitch. It's definitely the most popular project I've ever made; my friends think it's cool, and at $dayjob my current manager brought it up during my interview since he'd seen it on my website.+{{gallery>:birdbar:screenshots}}
  
-===== The Feeder =====+===== History ===== 
 + 
 +At the start of 2021 I received a window-mount bird feeder as a secret santa gift. As a bird lover I was excited to put it up and get a close up view of some of the birds that inhabited the woods around where I lived. Within around 3 days I had birds showing up regularly. 
 + 
 +With the floor plan of my apartment at the time, the only sensible place to put the feeder was on the kitchen window; there was a screened porch on my bedroom window, or I would have put it there. Since my work desk was in my bedroom, this meant that I couldn't watch it while I worked. If that had been an option, the rest of the project may never have materialized. 
 + 
 +Shortly after installing the feeder I had the idea to mount a camera pointing at it and stream it to Twitch, so that I could watch the birds while I was at my computer in another room. While watching I found myself wondering about a few of the species I saw and looking up pictures trying to identify them. Then it hit me - this is a textbook computer vision problem. I could build something that used realtime computer vision to identify birds as they appeared on camera. 
 + 
 +Fast forward a few years and this has bloomed into a pretty large project, with multiple upgrades to both the hardware, software and feeder setup. It's definitely the most popular project I've made; my friends think it's cool. It's also served as a good test bed to keep up to date on advances in machine learning and accelerated computing. 
 + 
 +===== Feeder =====
  
 This section covers the evolution of the feeder construction & installation details. This section covers the evolution of the feeder construction & installation details.
  
-With the floor plan of my apartment, the only sensible place to put the feeder was on the kitchen window; there’s a screened porch on my bedroom window, or I would have put it there. This meant that I couldn't watch it while I worked since my desk is in another room. If that had been an option, the rest of the project may never have materialized.+==== v1 ====
  
 Initially the feeder was mounted 'stock'. The camera was an old webcam I had lying around. Since it's mounted outside it needed to be weatherproofed. I did that with plastic wrap. Subsequent attempts greatly improved the design. Initially the feeder was mounted 'stock'. The camera was an old webcam I had lying around. Since it's mounted outside it needed to be weatherproofed. I did that with plastic wrap. Subsequent attempts greatly improved the design.
  
-{{:feeder-with-camera.jpg?400|Bird feeder showing webcam pointed at it}} {{:feeder-with-camera-outside.jpg?200|Picture of webcam attached to the side of my apartment building}} {{:webcam-condom.jpg?200|Picture of the webcam completely wrapped in plastic wrap secured by orange duct tape sitting on my window sill}}+{{birdbar:feeder:feeder-with-camera.jpg?400|Bird feeder showing webcam pointed at it}} {{:feeder-with-camera-outside.jpg?200|Picture of webcam attached to the side of my apartment building}} {{:webcam-condom.jpg?200|Picture of the webcam completely wrapped in plastic wrap secured by orange duct tape sitting on my window sill}}
  
 {{ :webcam-tupperware-lid.jpg?200|Picture of a tupperware lid taped over the camera as a sort of primitive rain shield}} {{ :webcam-tupperware-lid.jpg?200|Picture of a tupperware lid taped over the camera as a sort of primitive rain shield}}
Line 29: Line 39:
 ===== Bird Identification ===== ===== Bird Identification =====
  
-I’d read about [[https://pjreddie.com/darknet/yolo/|YOLO] some years before and began to reacquaint myself. It’s come quite far and seems to be more or less the state of the art for realtime computer vision object detection and classification. I downloaded the latest version ([[https://github.com/ultralytics/yolov5|YOLOv5]] at time of writing) and ran the webcam demo. It ran well over 30fps with good accuracy on my RTX3080, correctly picking out myself as “person”, my phone as “cell phone”, and my light switch as “clock”.+Birds arriving at the feeder are identified using [[https://github.com/ultralytics/yolov5|YOLOv5]] fine tuned on [[https://dl.allaboutbirds.org/nabirds|NABirds]]. 
 +==== Background ====
  
-{{:me-with-phone-yolo-detection.png?200|Screen capture of webcam feed after applying YOLOv5's out-of-box 'small' model to a scene of me holding up my cell phone. Picture is heavily blurred}}+I’d read about [[https://pjreddie.com/darknet/yolo/|YOLO]] some years before and began to reacquaint myself. It’s come quite far and seems to be more or less the state of the art for realtime computer vision object detection and classification. I downloaded the latest version ([[https://github.com/ultralytics/yolov5|YOLOv5]] at time of writing) and ran the webcam demo. It ran well over 30fps with good accuracy on my RTX3080, correctly picking out myself as “person”, my phone as “cell phone”, and my light switch as “clock”. 
 + 
 +{{:me-with-phone-yolo-detection.png?400|Screen capture of webcam feed after applying YOLOv5's out-of-box 'small' model to a scene of me holding up my cell phone. Picture is heavily blurred}}
  
 Out of the box YOLOv5 is trained on COCO, which is a dataset of _co_mmon objects in _co_ntext. This dataset is able to identify a picture of a Carolina chickadee as “bird”. Tufted titmice are also identified as “bird”. All birds are “bird” to COCO (at least the ones I tried). Out of the box YOLOv5 is trained on COCO, which is a dataset of _co_mmon objects in _co_ntext. This dataset is able to identify a picture of a Carolina chickadee as “bird”. Tufted titmice are also identified as “bird”. All birds are “bird” to COCO (at least the ones I tried).
  
-{{:chickadee-2.jpg?200|Image of chickadee with a poorly sized bounding box drawn around it with the label "bird" and a confidence rating of 0.31}}+{{:chickadee-2.jpg?400|Image of chickadee with a poorly sized bounding box drawn around it with the label "bird" and a confidence rating of 0.31}}
  
 Pretty good, but not exactly what I was going for. YOLO needed to be trained to recognize specific bird species. Pretty good, but not exactly what I was going for. YOLO needed to be trained to recognize specific bird species.
  
-===== Dataset =====+==== Dataset ====
  
 A quick Google search for “north american birds dataset” yielded probably the most convenient dataset I could possibly have asked for. Behold, [[https://dl.allaboutbirds.org/nabirds|NABirds]]! A quick Google search for “north american birds dataset” yielded probably the most convenient dataset I could possibly have asked for. Behold, [[https://dl.allaboutbirds.org/nabirds|NABirds]]!
Line 138: Line 151:
 After deciding that I wanted to maintain this as a long term installation I ponied up for a NUC and an eGPU enclosure. I initially tried to use the enclosure with an RTX 3070, but I couldn’t get it working with that card so I used a spare 1070 instead which worked flawlessly. The 1070 runs at about 25fps when inferencing with my bird model which is more than enough to look snappy overlaid on a video feed. The whole thing sits on my kitchen floor and is relatively unobtrusive. After deciding that I wanted to maintain this as a long term installation I ponied up for a NUC and an eGPU enclosure. I initially tried to use the enclosure with an RTX 3070, but I couldn’t get it working with that card so I used a spare 1070 instead which worked flawlessly. The 1070 runs at about 25fps when inferencing with my bird model which is more than enough to look snappy overlaid on a video feed. The whole thing sits on my kitchen floor and is relatively unobtrusive.
  
-====== 60fps ======+==== 60fps ====
  
 Up to this point I was streaming the window with annotated frames displayed by YOLO’s detect.py convenience script. However, this window updates only as often as an inferencing run completes, so around 25fps. It doesn't look good on a livestream. It would be better to stream video straight from the camera at native framerates (ideally 60fps) and overlay the labels on top of it. Up to this point I was streaming the window with annotated frames displayed by YOLO’s detect.py convenience script. However, this window updates only as often as an inferencing run completes, so around 25fps. It doesn't look good on a livestream. It would be better to stream video straight from the camera at native framerates (ideally 60fps) and overlay the labels on top of it.
Line 149: Line 162:
  
 This was stable for over a year, until I decided to install Windows 11. What could go wrong? This was stable for over a year, until I decided to install Windows 11. What could go wrong?
 +
 +==== Camera ====
 +
 +The original setup used an off-brand 720p webcam wrapped in a righteous amount of plastic wrap for weatherproofing. Surprisingly the weatherproofing worked well and there was never a major failure while using the first camera. However, the quality and color on that camera wasn’t good and an upgrade was due. I already had a Logitech Brio 4k webcam intended for remote work, but it ended up largely unused so it was repurposed for birdwatching.
 +
 +While the plastic wrap method never had any major failures it wasn’t ideal either. Heavy humidity created fogging inside the plastic that could take a few hours to wear off. It needed replacing anytime the camera was adjusted. Due to these problems and the higher cost of the Brio I decided to build a weatherproof enclosure.
 +
 +The feeder is constructed of acrylic. My initial plan was to use acrylic sheeting build out an extension to the feeder big enough to house the camera. I picked up some acrylic sheeting from Amazon and began researching appropriate adhesives. It turns out most adhesives don’t work very well on acrylic, at least not for my use case – the load bearing joints between the sheets were thin and I needed the construction to be rigid enough to support its own weight and the weight of the camera without sagging. Since the enclosure would be suspended over air relying on its inherent rigidity for structure the adhesive needed to be strong.
 +
 +The best way to adhere acrylic to itself is using acrylic cement. Acrylic cement dissolves the surfaces of the two pieces to be bonded, allowing them to mingle, and then evaporates away. This effectively fuses the two pieces together with a fairly strong bond (though not as strong as if the piece had been manufactured that way).
 +
 +{{:acrylic-cement.jpg?400 |}}
 +
 +{{:assembled-box.jpg?400 |}}
 +
 +Three sides were opaque to prevent sunlight reflections within the box. Joints were caulked and taped the joints to increase weather resistance. I played around with using magnets to secure the enclosure to the main feeder body but didn’t come up with anything I liked, so I glued it to the feeder with more acrylic cement, threw my camera in there and called it a day.
 +
 +{{ :full-setup-with-first-weather-shield.jpg?600 |}}
 +
 +This weatherproofing solution turned out great. It successfully protected the camera from all inclement weather until I retired that feeder, surviving rain, snow, and high winds over the course of the year.
  
 ====== Switching to Linux ====== ====== Switching to Linux ======
Line 190: Line 223:
 In the case of sexually dimorphic species that also have appropriate training examples, such as house finches, it’s even capable of distinguishing the sex. In the case of sexually dimorphic species that also have appropriate training examples, such as house finches, it’s even capable of distinguishing the sex.
  
-{{ :finches.mp4 |}}+{{ birdbar:screenshots:finches.mp4 |}}
  
 In a few cases, such as the nuthatch and the pine warbler, the model taught me something I did not know before. Reflecting on that, I think that makes this one of my favorite projects. Building a system that teaches you new things is cool. In a few cases, such as the nuthatch and the pine warbler, the model taught me something I did not know before. Reflecting on that, I think that makes this one of my favorite projects. Building a system that teaches you new things is cool.
Line 212: Line 245:
 Then I thought it would be cool to show these graphs on the livestream. It turns out Grafana supports embedding individual graphs, and since OBS supports rendering browser views it was easy to get those set up. Then I thought it would be cool to show these graphs on the livestream. It turns out Grafana supports embedding individual graphs, and since OBS supports rendering browser views it was easy to get those set up.
  
-{{:stats-display.png?400|screenshot from the stream with a bar graph showing species counts}}+{{birdbar:screenshots:stats-display.png?400|screenshot from the stream with a bar graph showing species counts}}
  
 I left these up for a while, but ultimately I felt they were taking up too much space in the stream so I took them down. I left these up for a while, but ultimately I felt they were taking up too much space in the stream so I took them down.
Line 225: Line 258:
   * Retrain with background images to reduce false positives   * Retrain with background images to reduce false positives
  
-{{:bluebird-and-chickadee.png?400|Eastern bluebird and Carolina chickadee feed together}}+{{birdbar:screenshots:bluebird-and-chickadee.png?400|Eastern bluebird and Carolina chickadee feed together}}
  
  
-{{tag>from_blog technology ai nature}}+{{tag>from_blog technology machine_learning nature}}
  
  
  
  
Panorama theme by desbest
bird_bar.1706746540.txt.gz · Last modified: by qlyoung · Currently locked by: qlyoung
CC Attribution-Noncommercial-Share Alike 4.0 International Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 4.0 International