ser: fix major slowdown by not doing excessive polls