The Boat Mill

Distance Off by Vertical Sextant Angle Calculator

Measure the vertical angle from the waterline to the top of a charted light or landmark, enter its height, and get your distance off in nautical miles — with the quick mental rule beside the exact answer.

When to use

Use this when piloting along a coast with a sextant (or a good hand-bearing inclinometer) and a charted object of known height whose base you can see at the waterline.

Take the height from the chart or light list. Charted heights are referenced to a stated datum (often mean high water in U.S. charts); if the water is lower than that datum, the object stands taller above the water — enter that difference as the height correction. Apply index error to the sextant reading before you enter it.

The method needs the base of the object at or inside your horizon. The page checks that against your height of eye.

Calculator

Light or landmark height from the chart.
Datum minus current tide; positive if the water is below the datum.
After index correction.
For the horizon check. 0 to skip.
—

How this is calculated

With the base of the object at the waterline in view, the object, its base, and your eye form a right triangle. The object’s height above the water is the side opposite the vertical angle, and your distance off is the adjacent side, so distance = height ÷ tan(angle).

For small angles, tan(angle) is close to the angle in radians, and one minute of arc is 1/3437.75 radian. That gives the quick rule: distance in nautical miles ≈ 0.566 × height in feet ÷ angle in minutes. The page shows both so you can see how close the mental arithmetic is.

height = charted height + correction
distance (ft) = height / tan(vertical angle)
distance (nm) = ft / 6076.12
quick rule: nm ≈ 0.566 × height (ft) / angle (minutes)
horizon check: nm ≈ 1.17 × √(height of eye, ft)

Worked checks

Each row below starts from the default inputs and changes only the columns shown. Recompute any row by hand with the formula above.

HeightCorr.DegMinFeetnmQuick nm
120003013,7512.262.26
120002020,6263.393.39
12000459,1671.511.51
80001518,3353.023.02
16041108,0531.331.33
5200404,4690.740.74

Step by step: the default inputs

With the calculator defaults (Charted height of object (ft) = 120; Height correction (ft) = 0; Vertical angle — degrees = 0; Vertical angle — minutes = 30; Height of eye (ft) = 8), the arithmetic runs like this:

height used = 120 + 0 = 120 ft
angle = 0° 30′ = 30′ = 0.008727 rad
distance = height / tan(angle) = 120 / 0.008727 = 13,751 ft
nautical miles = ft / 6076.12 = 2.263 nm
horizon ≈ 1.17 × √eye = 1.17 × √8 = 3.31 nm

Result: 2.26 nm distance off. 13,751 ft (4.19 km). Quick rule 0.566 × height ÷ minutes = 2.26 nm. Inside your 3.3 nm horizon.

Second example: one input changed

Now change Vertical angle — minutes to 20 and leave everything else alone:

height used = 120 + 0 = 120 ft
angle = 0° 20′ = 20′ = 0.005818 rad
distance = height / tan(angle) = 120 / 0.005818 = 20,626 ft
nautical miles = ft / 6076.12 = 3.395 nm
horizon ≈ 1.17 × √eye = 1.17 × √8 = 3.31 nm

Result: 3.39 nm distance off. 20,626 ft (6.29 km). Quick rule 0.566 × height ÷ minutes = 3.39 nm.

A smaller angle means you are farther off: dropping from 30′ to 20′ moves the fix from about 2.26 to 3.39 nm. Because distance is inversely proportional to the angle, small reading errors matter most at small angles — take two or three sights and average them.

Navigation notes

A distance off is a circle of position, not a fix. Cross it with a bearing of the same object, or with a second distance off another object, and you have a position. A bearing plus a vertical-angle distance from one lighthouse is one of the quickest fixes in coastal piloting.

Use the height of the light, not the tower, when sighting the light at night — and the top of the structure, not the light, by day if that is what the chart gives. Read the light list carefully: some heights are focal-plane heights.

Keep the sextant reading honest: correct for index error, and bring the top of the object down to the waterline at its base, not to the horizon behind it. If the base is hidden by the horizon or by land, this formula is the wrong one.

Limits / assumptions

Units: Feet for heights, degrees and minutes for the angle, nautical miles and feet for distance.

Where the model stops: Base of the object visible at the waterline, flat-sea right-triangle geometry. Refraction, beyond-horizon objects, and a raised shoreline base are not modeled.

Not a substitute: Cross-check with other fixes, radar, and GNSS, and follow your chart and light list for heights and datums.

Distance off (nm) by charted height and angle

Angle50 ft120 ft300 ft600 ft
10′2.836.7916.9733.95
20′1.413.398.4916.97
30′0.942.265.6611.32
45′0.631.513.777.54
60′0.471.132.835.66
90′0.310.751.893.77

Values are from the exact formula used by the calculator. The quick rule (0.566 × height ÷ minutes) agrees to within a few hundredths of a mile at these small angles.

Getting a good angle

FAQ

Do I need a sextant?

A sextant gives the precision this method needs. Inclinometers and phone apps can work for rough checks, but a one-minute error at 30′ is a 3% distance error.

Which height do I use?

The charted height of the object above its datum, corrected for the current tide if the water is below that datum. Add the correction in the second field.

Why does the page warn about the horizon?

If you are farther away than your horizon, the base of the object is hidden below it. Measuring to the visible horizon instead of the base needs a different formula.

How accurate is the quick rule?

Very close at small angles. In the default example the quick rule and the exact answer agree to 0.01 nm.