Fitness trackers aren’t very accurate when it comes to measuring heart rate, especially if you’re doing a hard workout, researchers say. This could be risky for people who might push themselves too hard, and frustrating for those who wonder why a short jog makes their heart rate skyrocket.

Researchers recruited 50 healthy volunteers to test four popular fitness trackers for a study published this week in JAMA Cardiology. They connected volunteers to an electrocardiogram (EKG), which is the most accurate way of measuring heart rate. Then, volunteers wore the trackers — the Apple Watch, Mio Fuse, Fitbit Charge HR, and Basis Peak — while exercising on a treadmill at 2, 3, 4, 5, and 6 miles per hour.

The Apple Watch and Mio were the most accurate, the study says. They matched the EKG about 91 percent of the time, though the Apple Watch was a bit more consistent. The Fitbit Charge HR was accurate 84 percent of the time, and the Basis Peak was accurate 83 percent of the time.

The researchers found that the harder someone exercised, the less accurate the trackers were. Fitbit tended to underestimate the heart rate, while the Basis overestimated it. “At rest, all of the monitors did an acceptable job,” says study co-author Gordon Blackburn, a cardiac researcher at the Heart and Vascular Institute at Cleveland Clinic. “As the intensity of the exercise went up, we saw more and more variability in the accuracy.”

Fitness trackers usually measure heart rate using sensors that detect the pulse through the skin on your wrist. But when you exercise vigorously, your arms tend to swing. This makes the tracker slide up and down, which makes the reading less accurate. In addition, some of the models seemed to be losing connection when the sensors would get too far from the skin.

Accuracy starts to fall around 100 beats per minute, and becomes very inconsistent above 130 and 140 beats per minute, Blackburn says. These are realistic ranges for healthy people to shoot for while exercising, so this inaccuracy is a real problem. “Patients come in with data and they say, ‘I’m exercising and my heart rate is going sky high,’ or ‘I can’t get into the zone’ and they’re getting frustrated,” he says. “The people developing these trackers say it’s not a medical technology, but that message isn’t being absorbed by users. And we’re caught in the middle when they come into the office presenting data and we have to convince them there is significant error.”

The biggest health risk is when a tracker underestimates the heart rate, like the Fitbit does. (In fact, Fitbit is being sued for inaccurate heart-rate monitoring as part of a class action lawsuit.) Underestimating can lead people to push themselves beyond what’s healthy, especially if they’re not in great shape. Still, overly high readings can make people slow down their exercise when there is no need to.

A spokesperson for Fitbit emphasized that the trackers are not meant to be medical devices, adding that the company tested its trackers internally and they had an average error of less than 6 beats per minute.

There are some limitations to the study. First, the researchers only used the treadmill. All the participants were relatively healthy, and most of the participants were white. Now, the team is conducting follow-up studies, this time looking at different types of exercise (ellipticals, biking) and other fitness trackers.

Update: This article has been updated to include a statement from Fitbit.