Many thanks to Andrei of School 205 Bucharest for the inspiration for this solution. Well done Andrei. One might have expected the triangular solution to be best. I wonder what happens if you chop off the corners?

Using the following notation:

h - for the height of the cup and

d - for the diameter of the cup.

Test each of the two possible forms of the box, one with a rectangular base and one with a triangle as its base:

Cuboidal box

As there are six cups they could be ordered in two ways:

1 x 6 cups

and

2 x 3 cups

For any cuboid box, the surface area is: 2 times the top area + 2 times the side area + 2 times the front area:

The total area of a 1 x 6 cups box is:

2 x (h x 6 x d + d x 6 x d + d x h) = 185470 mm2

The total area of a 2 x 3 cups box is:

2 x (h x 2 x d + h x 3 x d + 2 x d x 3 x h) = 157250 mm2

For a triangular prism box This case is illustrated in the figure of the problem.



The base is an equilateral triangle. The surface area is 2 times the area of the equilateral triangle + 3 times the area of each of the faces.

First calculate the side of the equilateral triangle ABC.

This is 2 x BM + 2 x 85 mm.

BM is a tangent to the circumference of a corner cup (centre O, radius d/2 = 85/2 mm).

Also the angle MOB = 60°

Therefore angle MBO = 30°


BM
= 85
2
 tan60°
  
= 85
2
 Ö3
  
= 85Ö3
2
The side of triangle ABC is
= 2 x d + 2 x BM
  
= 2 x 85 + 2 x 85Ö3
2
  
= 85(2 + Ö3)
The altitude of the equilateral triangle ABC is
= AB x sin60°
  
= 85(2 + Ö3) x Ö3
2
  
= 85Ö3(2 + Ö3)
2
Area of the two equilateral triangles
= 85(2 + Ö3) x 85Ö3(2 + Ö3)
2
  
= 87149 mm2
Area of the three faces
= 3 x 83 x 85(2 + Ö3)
  
= 78989 mm2
The total surface area is
= 166138 mm2
So the best box is a box of 2 x 3 cups.