When trying to expand some columns in a table where the sum of the

widths of the remaining columns is already wider than the line
length, underflowing size_t and dying from ENOMEM is the wrong plan.
Instead, simply refrain from expanding anything in such a situation,
avoiding a crash that tb@ found with afl.

4 files changed, 46 insertions, 5 deletions

diff --git a/regress/usr.bin/mandoc/tbl/mod/Makefile b/regress/usr.bin/mandoc/tbl/mod/Makefile
index c5bebb0ff5f..ab087b1e64c 100644
--- a/regress/usr.bin/mandoc/tbl/mod/Makefile
+++ b/regress/usr.bin/mandoc/tbl/mod/Makefile
@@ -1,6 +1,6 @@
-# $OpenBSD: Makefile,v 1.2 2015/02/10 11:02:19 schwarze Exp $
+# $OpenBSD: Makefile,v 1.3 2017/05/01 20:53:58 schwarze Exp $
 
-REGRESS_TARGETS	 = badfont expand font misalign
+REGRESS_TARGETS	 = badfont expand expand-toowide font misalign
 LINT_TARGETS	 = badfont font
 
 # groff-1.22.3 defects:
diff --git a/regress/usr.bin/mandoc/tbl/mod/expand-toowide.in b/regress/usr.bin/mandoc/tbl/mod/expand-toowide.in
new file mode 100644
index 00000000000..7e8a44b81c0
--- /dev/null
+++ b/regress/usr.bin/mandoc/tbl/mod/expand-toowide.in
@@ -0,0 +1,15 @@
+.TH TBL-MOD-EXPAND-TOOWIDE 1 "May 1, 2017" OpenBSD
+.SH NAME
+tbl-mod-expand-toowide \- try to expand columns in a table
+that is already too wide
+.SH DESCRIPTION
+.ll 20n
+In this section, the line length is limited to 20 characters.
+.TS
+tab(:);
+lx l lx.
+x:1234567890:x
+.TE
+.ll 78n
+.PP
+Now we are back to the default terminal width.
diff --git a/regress/usr.bin/mandoc/tbl/mod/expand-toowide.out_ascii b/regress/usr.bin/mandoc/tbl/mod/expand-toowide.out_ascii
new file mode 100644
index 00000000000..066fdf90d5c
--- /dev/null
+++ b/regress/usr.bin/mandoc/tbl/mod/expand-toowide.out_ascii
@@ -0,0 +1,23 @@
+TBL-MOD-EXPAND-TOOWIDE(1)   General Commands Manual  TBL-MOD-EXPAND-TOOWIDE(1)
+
+
+
+NNAAMMEE
+       tbl-mod-expand-toowide - try to expand columns in a table that is
+       already too wide
+
+DDEESSCCRRIIPPTTIIOONN
+       In this
+       section, the
+       line length
+       is limited to
+       20
+       characters.
+
+       x   1234567890   x
+
+       Now we are back to the default terminal width.
+
+
+
+OpenBSD                           May 1, 2017        TBL-MOD-EXPAND-TOOWIDE(1)
diff --git a/usr.bin/mandoc/out.c b/usr.bin/mandoc/out.c
index 5e48487e90a..8d483a6f3bb 100644
--- a/usr.bin/mandoc/out.c
+++ b/usr.bin/mandoc/out.c
@@ -1,7 +1,7 @@
-/*	$OpenBSD: out.c,v 1.34 2015/10/12 00:07:27 schwarze Exp $ */
+/*	$OpenBSD: out.c,v 1.35 2017/05/01 20:53:58 schwarze Exp $ */
 /*
  * Copyright (c) 2009, 2010, 2011 Kristaps Dzonsons <kristaps@bsd.lv>
- * Copyright (c) 2011, 2014, 2015 Ingo Schwarze <schwarze@openbsd.org>
+ * Copyright (c) 2011, 2014, 2015, 2017 Ingo Schwarze <schwarze@openbsd.org>
  *
  * Permission to use, copy, modify, and distribute this software for any
  * purpose with or without fee is hereby granted, provided that the above
@@ -195,9 +195,12 @@ tblcalc(struct rofftbl *tbl, const struct tbl_span *sp,
 	 */
 
 	if (nxcol && totalwidth) {
-		xwidth = totalwidth - xwidth - 3*maxcol -
+		xwidth += 3*maxcol +
 		    (opts->opts & (TBL_OPT_BOX | TBL_OPT_DBOX) ?
 		     2 : !!opts->lvert + !!opts->rvert);
+		if (xwidth >= totalwidth)
+			return;
+		xwidth = totalwidth - xwidth;
 
 		/*
 		 * Emulate a bug in GNU tbl width calculation that