When it comes to losing weight, most of us follow a simple formula: burning more calories + eating fewer calories = weight loss. Exercise is one way we try to burn more calories, so we hit the gym or pick up a pair of weights thinking we’ll eventually see the number on the scale inch its way down. Unfortunately, it doesn’t always work that way, something that often frustrates new exercisers. If you’re doing all this exercise, you should be losing weight, right?

The truth is, exercise is a complicated business and there are a number of things that can affect how many calories you burn. Knowing what those are will help you set realistic goals and get the most out of your workouts.

If you’re trying to lose weight with exercise, you may have used an activity calculator to determine how many calories you’re burning. For example, if you’re 165 lbs and you go jogging for 30 minutes, this calculator shows you’ve burned about 371 calories. Not bad for a 30-minute workout, you might think, but are you getting the whole story? Not exactly. There are a few other things to consider when it comes to exercise and weight loss.

1. Net Calories vs. Gross Calories

Most calculators use activity, duration of your workout and your weight to come up with an estimate of calories burned, or what is known as gross calories burned. What we forget to factor in are the calories we would’ve burned if we weren’t exercising, also known as the net calories burned.

If you jogged during a time you normally watch TV, you’re still burning more calories than you were, but you need to subtract the calories you would’ve burned while watching TV in order to get a more accurate calculation.

It may seem like a small difference, after all, you may burn more than 300 calories jogging and only about 40 calories watching TV.

This difference becomes important, however, when you’re trying to predict weight loss. Those 40 calories, if unaccounted for, can add up to fewer pounds lost.

What you can do: If you’re tracking calories burned with exercise, you’ll get a more accurate number by subtracting the calories you would’ve burned if you weren’t working out. For example, if you burned 200 calories while walking for 20 minutes and would’ve burned 50 calories if you sat at the computer during that time, your net calories burned would be 150. You can calculate your calories with an activity calculator.

2. Exercise Intensity

You probably know that taking a leisurely stroll won’t burn as many calories as, say, running a mile as fast as you can. How hard you work plays a role in how many calories you’re burning. Some calculators, especially those on cardio machines like treadmills and elliptical trainers, do take into account things like pace, resistance, and incline. We also know the relative intensity of a number of activities, but using this information to estimate how much weight you’ll lose is tough.

For example, if you were to burn 2,000 calories a week with a walking program, you might expect to lose about 6 pounds of fat after 10 weeks of exercise.

The problem is, this assumes you burned exactly 2,000 calories each week and that 6 pounds of fat would generate exactly 6 pounds of body weight loss, which isn’t always the case.

What you can do: The formulas we use to calculate exercise intensity and calories burned aren’t 100% accurate. Rather than rely solely on those numbers, learn how to monitor your intensity using the talk test, perceived exertion and/or target heart rate. You’ll find your own limits while keeping track of how hard you’re working. You can get the most out of your workouts by:

• Varying your intensity: The harder you work, the more calories you burn, but if all your workouts are high intensity, you run the risk of overtraining and injury. By incorporating a variety of intensity levels, you’ll stimulate different energy systems while giving your body a break from too much high-intensity exercise. Interval training is a great option for working harder while still getting some rest time.

• Using a heart rate monitor: A heart rate monitor is a great tool for getting an accurate view of your heart rate throughout your workout while keeping you on pace. Many monitors also show calories burned during your workout and you can use that number to compare different workouts and different intensity levels.

3. The Type of Workouts you Do

While any exercise is good for the body, some activities do burn more calories than others. Weight-bearing activities like running, aerobics or walking, burn more calories because gravity requires your body to work harder. When doing non-weight-bearing exercises like cycling or swimming, there isn’t as much gravitational stress on the muscles, which means fewer calories expended.

What you can do: Non-weight-bearing activities do have advantages. They’re less stressful on the joints and you can often do them longer, which could make up the difference in calories burned with weight-bearing activities. However, cross-training with impact activities, if you’re able to do that, will not only work your body in different ways, it also helps build stronger bones and connective tissue.

4. Mechanical Efficiency

You probably never thought that being good at an activity would mean burning fewer calories, but that’s exactly what happens when you exercise consistently. Think about the first time you tried a treadmill or some other cardio machine. You probably felt awkward, holding onto the rails and worrying you might fall off. Over time, the movement became so natural, you didn’t have to think about it anymore. As your body became more efficient, you stopped wasting energy on unnecessary movements, which leads to fewer calories burned.

What you can do: Mechanical efficiency is actually a good thing. By cutting down on awkward movements, your body works more efficiently, which helps protect you from injury.

5. Exercise Compensation

Something else we don’t often consider is how exercise affects our activity for the rest of the day. If you do a tough workout and then take a nap or skip an afternoon walk, something you wouldn’t normally do, you’re burning fewer calories. Exercise can also increase your appetite, causing you to eat more calories which can also sabotage your weight loss goals.

What you can do: If you start an exercise program, keep a food and activity journal to get an idea of a normal day for you. Resting more or eating more are things we often do without being aware of it post-exercise. Keeping a simple log of your activities will help you make sure you’re getting the same amount of activity, even with your workouts. If you’re exhausted after every workout, that may be a sign you’re overdoing it. You want to keep a little gas in the tank after most of your workouts.

6. Body Mass

Another irony with weight loss is that, the heavier you are, the more calories you’ll burn with exercise. For example, a 200-lb pound person can burn about 400 calories during 30 minutes of stair climbing, while a 125-lb person burns about 250 calories doing the same thing. As you lose weight, your body expends less energy to move your body around, which means you’ll lose weight more slowly. This is one reason why losing those last few pounds can be so difficult.

What you can do: First, remember that losing weight is a good thing, even if it means the weight loss slows down over time. Second, as you lose weight, you may need to recalculate how many calories you need and how many calories you’re burning. Adjusting the numbers as you go can help you stay on track with your weight loss and avoid plateaus.

7. Genetics and Gender

While we control many of the factors involved with weight loss, there are some things we really can blame on our parents: Genes and gender. Our genes often determine the resting metabolic rate, muscle fiber types and genetic responses to different foods, all of which can affect our ability to burn calories and lose weight. Your body type does play a role in your ability to lose weight, as does your lifestyle.

Gender can also affect weight loss. Women usually have more body fat than men and their bodies respond differently to exercise, which can change the rate of weight loss.

What you can do: Recognize that there may be genetic factors that affect how quickly you lose weight. While you might inherit certain genes from your parents, your eating and exercise habits can make a difference. The only way to know what your body is really capable of is to try. Following a complete exercise program  and watching your calories is the best way to find out what your body can really do.

It can be frustrating when you start exercising and don’t see the results you’re expecting. You may start wondering: “What am I doing wrong?” Knowing there are other factors involved can help you be more realistic and, perhaps, encourage changes to your workouts to get more out of your program. If you’re distracted and discouraged by calculations that don’t seem to add up, remember that they’re only numbers. They don’t reflect the tangible results you may be experiencing such as feeling better, having more energy or simply having a better day-to-day life.