I'm sorry to report what you're trying to do with the boat battery idea probably won't work up to your expectations for very long - but I do have a suggestion if you still wanted to go that route (although doubling the current battery with the same type battery like thedeuceman suggested might be a better option (provided you connect them balanced) since it is obviously simpler - although just doubling a smaller battery still might not be able to keep up with the winch consumption, especially if the alternator is small - so then a larger capacity battery that could be recharged between uses would still be your best bet). These are just suggestions based on my own experiences since I've worked with many types of inverters and various types of battery combinations and configurations since the 80's. I'll try to keep this as short as possible and realize that it's probably way more than you'd even care to know about, but it's more complicated than people sometimes give battery configuration technology credit for - especially when in numerous examples everyday even inefficient configurations have met their owners expectations for years (I have a neighbor who throws a small CHEAP non-deep cycle battery in his camper every couple of years who is perfectly satisfied with what would seem the MOST wrong way to do it - but he's plugged in everywhere (boils it to death) and it meets his every need and expectation because he has never taxed it's capacity beyond maybe 10 percent and it's inexpensive to replace...hard to argue with that (until he freezes some night when the power goes out).
First, assuming you were to use the "hook the boat battery to the engine battery parallel idea", the reason I suggest it won't work for long is because you're trying to combine unlike and different size batteries, second reason is that the suggest wiring would likely be too small to carry the additional charge - plus the way you are proposing to hook it up is not balanced, so the current would not pull equally from both batteries (although with two greatly different capacity batteries it won't pull evenly regardless). Even some new motorhomes and boats are not balanced right from the manufacturer to this day yet (a personal pet peeve of mine)...for example; with just two batteries paralleled, you have to hook the load to the positive of one battery and the negative to the other battery - otherwise most of the load comes off the first battery and the second battery acts more like a semi-recharging "jump start" arrangement. Our last motorhome (plus countless others I've seen) came with this arrangement and the inverter would only run the microwave for about two minutes if both batteries were fully charged even though I had installed oversized wiring for the expected load, but by simply balancing, it can now run it for around 20 minutes with only that simple change (yes, I've tested).
Of course, while less efficient, I'm sure there will be many who would say the unbalanced method is working just fine for them (although I'll bet they wind up replacing their "worn out" batteries much sooner than they need to)...it all boils down to how much current you are drawing at one time and if that second battery (plus alternator) can keep up with "recharging" the first one, which if your amperage (current) needs are small it usually can - but in your case, you are pulling large momentary amps for the winch and trying to recharge with probably a very small alternator which is no doubt a taper charger to boot (throttles current amps based on voltage rather than state of charge (SOG), in other words, you'd have to run your engine for many more hours to get a complete charge). Plus, because they are "combined" continuously, the batteries are always going to try to equalize even when there is no charge source occurring. There is almost no way this uneven match arrangement will work without killing your added boat battery long before your expectations, especially if you intend to use it again for the following year's boating. You could add a small "battery tender" if you have enough time in between uses, although you'd be better off adding a three stage charger so that it gets a full charge in between uses because both over and/or under charging will shorten it's lifespan greatly.
My suggestion for you (assuming you'd still want to use the added boat battery idea) is to use something that (believe it or not) Fleet Farm carries now. It's a battery combiner that only connects them when one or the other are receiving a charge. You would hook your cables from your second battery straight to your winch - completely separate from your other battery (do that whether you use this combiner or not). When the engine is running, if there is enough charge or voltage from the alternator or small battery, it will give the extra to the winch (boat) battery. In between uses, if you have a battery tender or (preferably) three stage charger on the (I would recommend) winch battery, it will combine once that battery is charged enough to maintain both batteries. I use this same arrangement (with that inexpensive "Fleet" combiner) on my boat because I have one battery for a trolling motor & stereo - although my use is to keep one from discharging the other rather than trying to increase capacity like you are trying to accomplish. This combiner eliminates the disadvantage of the diode type unit's voltage drop too.
One final caution though - if your boat battery is a different "type" such as your engine battery being lead acid and your boat battery is AGM or gell - you'd probably be better off just adding a charger to only your winch (boat) battery along with running the cables from that battery to the winch and not combining them to your engine battery at all because of the difference in charging requirements. The difference is slight, but it has a big impact on the battery life if done wrong.
As I said, just suggestions based on "school of hard knocks" experience. If this seems like overkill and much more than you want to do but still want to use you boat battery - as a minimalist approach I'd suggest getting the cables (large enough for the load and extra length voltage drop of course) and hook it up in place of the current engine battery (take that one out of the loop completely), that way you eliminate the capacity mismatch all-together. You still may need to add a separate plug-in charger for in-between uses if (almost guaranteed if the system was designed for a smaller battery, but it still will increase the time you're able to run before being forced to plug it in) the alternator can't keep up (plus avoiding the life-shortening undercharge), but at least that mismatch between batteries wouldn't kill your boat battery for the next boating season. Good luck with whichever way you decide to go!
