An Auto Layout Adventure: NSCell, -intrinsicContentSize, and -constraintsAffectingLayoutForOrientation:

Recently my coworker Tom was having a hard time with converting a Mac NIB to Auto Layout. The NIB contained a split view; on the left was an instance of OACalendarView, and on the right was a scroll view. The holding priorities of the left and right panes were 251 and 250, respectively, and the scroll view had a required width constraint of greater-than-or-equal-to 150.

For some reason, the left pane insisted on being as wide as possible, squeezing the right pane down to 150 points wide. Dragging the splitter had no effect. How do we figure this one out?

The first debugging step was to enumerate which constraints were being applied to the subviews. NSView has a method called -constraintsAffectingLayoutForOrientation: that returns an array of constraints that affect layout of the receiver on the specified axis. There’s also a fantastic interface for inspecting constraints at runtime: pass an array of constraints to -[NSWindow visualizeConstraints:] and they’ll be drawn atop your interface.

These two debugging features are so powerful that I built a convenience into our frameworks that, in response to the keystroke Command-Control-Option-Shift-V, lets you pick a view to visualize its vertical or horizontal constraints.

Unfortunately, that wasn’t of any help. The documentation for -constraintsAffectingLayoutForOrientation: warns that it may not return all the constraints involved in laying out a specific view, and that was certainly the case here. There was no reason for the constraints it did return to cause the behavior we were seeing.

We were stumped, and I begrudgingly abandoned the problem.

A few days later, Tom informed me that he’d fixed the problem by removing OACalendarView‘s implementation of +cellClass. Thus, instead of -cell returning an instance of NSActionCell, it was left as nil. The fact that this worked was only slightly more puzzling than why the calendar view had a cell in the first place.

OACalendarView is a subclass of NSControl, and thus has a target and an action. It sends its action when the selected day changes. But while a typical NSControl uses one cell instance to draw its entire contents, OACalendarView is more like NSTableView in that it stamps out the same image repeatedly. The only reason OACalendarView had a value assigned to its cell property was so that it fall back on NSControl‘s implementation of -target and -action, which just query the control’s cell. Without a cell, NSControl returns nil from these methods, and -setTarget: and -setAction: have no effect. Even though the target and action properties are declared on NSCell, as an artifact of history they are only supported on instances of NSActionCell.

Hence the override of +cellClass, which exists as a convenience for setting the cell of an object that’s loaded from a NIB. You can drag a Custom View object from the library and drop it into your NIB, change its class on the Identity Inspector, and at runtime the control will call [self setCell:] with an instance of the class returned by +cellClass.

Removing +cellClass eliminated the bad layout behavior at the expense of the target/action support. Clearly there was something strange about NSActionCell, but in the meantime Tom replaced OACalendarView‘s use of -target and -action with a delegate protocol and was quite happy to have a working UI again.

Last night, I learned why +cellClass was the culprit.

My current task involves subclassing a few different NSCell subclasses to do some custom drawing. Since that involves rearranging parts, I need to ensure that the views using those cells return the correct values from -intrinsicContentSize. As pointed out to me by Ken Ferry, and as documented alongside -invalidateIntrinsicContentSizeForCell:, -[NSControl intrinsicContentSize] defers to its cell’s -cellSize.

This got me thinking. As another artifact of history, NSCells can come in three flavors: text cells (cells that were initialized with -initTextCell:), image cells (those initialized with -initImageCell:) and “null” cells (those initialized with -init). (Nowadays you’d probably just create an instance of either NSTextFieldCell or NSImageCell, whose overrides of -init call the correct variant on their superclass.) -cellSizeForBounds: has meaningful return values for text and image cells, but what for null cells?

It turns out for null cells, -[NSCell cellSizeForBounds:] just returns the size of its argument. -[NSCell cellSize] simply calls -cellSizeForBounds: with a really big rectangle (40,000 points on a side). And since OACalendarView relied on its superclass automatically creating an instance of its +cellClass, the cell was being initialized using -init, and thus it was a null cell.

So here’s what was happening: the content compression resistance priority of the calendar control was left at the default of 750. This is higher than either of the split view pane’s holding priorities. When Auto Layout asked the calendar control for its -intrinsicContentSize, it consulted with its cell, who happily replied that it wanted to be 40,000 points wide. Thus, the calendar pushed everything else out of the way as it tried to fill the window. -constraintsAffectingLayoutForOrientation: omitted the internal content compression resistance constraint, so we never saw it when invoking the constraint debugging interface.

As much as I love Mac programming, iOS developers should be grateful that they don’t have to deal with the legacy of NSCell.