From patchwork Fri Sep 27 17:54:22 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Lance Taylor X-Patchwork-Id: 278658 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 1F2F92C0339 for ; Sat, 28 Sep 2013 03:54:38 +1000 (EST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:date:message-id:mime-version:content-type; q=dns; s= default; b=XKpUDXA9oSZeQOMrHOzO9zYjKOFgNUliFZL4AtmUhlzQtnU9SFybu bH5iBT7C/0+xcNsf4xqYzvUvqIQMTIOfJyv1pM74LFJgYYfFwVH7AOtp53p3pIPz OagY5Ff/LEAe91f0S67BzcFnRooyDfG74cLsxhLMGsU16Sizjn6Zcw= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:date:message-id:mime-version:content-type; s= default; bh=Ym5n0xvFfgpsgjeSkiRMi+zZRFI=; b=hrnYthBw9TnWG8fe5rTB xWV+dHFrCWQZ3eYXxzAV+BIRxOzBndWfrvd62Yg7kPUVtTvca8Vbc0rr/wxYWHcP GRTqhId/pCZT9LETE4d4gVU9z12/pmifj9szI1FlK+k+oQHP/0MQkhz+1Q/ATZV+ CKMLA2i91SH6L174UzQqfu4= Received: (qmail 7690 invoked by alias); 27 Sep 2013 17:54:30 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 7678 invoked by uid 89); 27 Sep 2013 17:54:29 -0000 Received: from mail-pa0-f43.google.com (HELO mail-pa0-f43.google.com) (209.85.220.43) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES128-SHA encrypted) ESMTPS; Fri, 27 Sep 2013 17:54:29 +0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=1.5 required=5.0 tests=ALL_TRUSTED, AWL, BAYES_50, SPAM_SUBJECT, T_TVD_MIME_NO_HEADERS autolearn=no version=3.3.2 X-HELO: mail-pa0-f43.google.com Received: by mail-pa0-f43.google.com with SMTP id hz1so3084229pad.2 for ; Fri, 27 Sep 2013 10:54:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id:user-agent :mime-version:content-type; bh=tO4sqFytj4Q0uqp+tVtTnSoJWL0IwaQIL7bvw9xFETQ=; b=jPdhC6zpqWqRTvlO6JjamCopox5t7fXXXxxSLeUCSalmUZXRCS3w7Bp+E9fpZ1cDRL SANGH/Of/cfqR8zCRerCb6RpT7fws00hWGWu/AUy6I0bkeTjYSbCoinAm80fkfxoreg0 G2X4IzD3uafqPFcMvgojaPmguDdDp5birw3Uhh1KypIVSApEf/8B0p7RRzTZgiHEoeac 1S2e8y9MoAaGsM4x81xhoR1Lxw5AJhQfew10/7Grm5qZNB6jP+3u/pSGtp8Xaf88XXLU ePqVXCattNwaMOjlpJ8vhlsZR00LCkai3eZFnnKMfZcdHMip0WXJe0uDTNWp7amjm78j H+7Q== X-Gm-Message-State: ALoCoQnMIDCI74YOW0P8X+C14ZKWIeDkiHdtue8eWz7oSD8HnF50Sx/ZnuUw5DITXaiW7WJ15s0CGAptBIC++q1/ZxPffiAxtaI2h0MVNGWuT9r8ZT+FVFMjnRvxtp6DGylkua5SfJwHZAgwacYkn0y45csmMmG2O3Ed2mJ1x7s6LVdPDEYzaGSGTZH3ptLJqtylcx1GuYhLlLAoBSlyY25vEa60XVlrcw== X-Received: by 10.68.202.38 with SMTP id kf6mr8807243pbc.43.1380304465893; Fri, 27 Sep 2013 10:54:25 -0700 (PDT) Received: from iant-glaptop.roam.corp.google.com.google.com ([2620:0:1000:147c:691e:dd6e:de18:4491]) by mx.google.com with ESMTPSA id py4sm10115749pbb.33.1969.12.31.16.00.00 (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Fri, 27 Sep 2013 10:54:25 -0700 (PDT) From: Ian Lance Taylor To: gcc-patches@gcc.gnu.org, gofrontend-dev@googlegroups.com Subject: libgo patch committed: Implement reflect.MakeFunc for amd64 Date: Fri, 27 Sep 2013 10:54:22 -0700 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (gnu/linux) MIME-Version: 1.0 X-IsSubscribed: yes The Go standard library has an interesting function named reflect.MakeFunc. It takes a Go function F that accepts and returns a slice of reflect.Value, and a function type T, and returns a pointer to a function of type T that converts its arguments to reflect.Value, calls F, and converts the returned reflect.Value into the appropriate return types. In effect this is the reverse of libffi: instead of describing a function and calling it, we describe a function and permit it to be called. For gccgo I tried to implement this generically using the builtin varargs functions, but that failed because I had no way to handle the return type. Many Go functions return multiple values, which in gccgo is represented as returning a struct, and, of course, in some cases a struct is returned by passing a hidden pointer as the first argument, and in other cases is handled by splitting up the struct into different register classes. So handling this generically is essentially impossible, at least without adding some more builtin functions to somehow handle the return value, builtin functions that I couldn't figure out how to even represent. So I gave up and went for a processor-specific approach. The idea is that processor-specific assembly code will save all the relevant registers into a struct, and pass them to processor-specific Go code which will implement the calling convention. This has the advantage that I only need to deal with Go types, which in particular means no worries about vector types. This patch implements this approach for x86_64. Bootstrapped and ran Go testsuite on x86_64-unknown-linux-gnu. Committed to mainline and 4.8 branch. Ian diff -r 024105249263 libgo/Makefile.am --- a/libgo/Makefile.am Tue Sep 24 20:26:38 2013 -0700 +++ b/libgo/Makefile.am Fri Sep 27 08:06:13 2013 -0700 @@ -895,9 +895,21 @@ go/path/match.go \ go/path/path.go +if LIBGO_IS_X86_64 +go_reflect_makefunc_file = \ + go/reflect/makefuncgo_amd64.go +go_reflect_makefunc_s_file = \ + go/reflect/makefunc_amd64.S +else +go_reflect_makefunc_file = +go_reflect_makefunc_s_file = \ + go/reflect/makefunc_dummy.c +endif + go_reflect_files = \ go/reflect/deepequal.go \ go/reflect/makefunc.go \ + $(go_reflect_makefunc_file) \ go/reflect/type.go \ go/reflect/value.go @@ -1761,6 +1773,7 @@ os.lo \ path.lo \ reflect-go.lo \ + reflect/makefunc.lo \ regexp.lo \ runtime-go.lo \ sort.lo \ @@ -2147,6 +2160,9 @@ $(BUILDPACKAGE) reflect/check: $(CHECK_DEPS) @$(CHECK) +reflect/makefunc.lo: $(go_reflect_makefunc_s_file) + @$(MKDIR_P) reflect + $(LTCOMPILE) -c -o $@ $< .PHONY: reflect/check @go_include@ regexp.lo.dep diff -r 024105249263 libgo/go/reflect/all_test.go --- a/libgo/go/reflect/all_test.go Tue Sep 24 20:26:38 2013 -0700 +++ b/libgo/go/reflect/all_test.go Fri Sep 27 08:06:13 2013 -0700 @@ -1430,11 +1430,13 @@ } } -/* - -Not yet implemented for gccgo. - func TestMakeFunc(t *testing.T) { + switch runtime.GOARCH { + case "amd64": + default: + t.Skip("MakeFunc not implemented for " + runtime.GOARCH) + } + f := dummy fv := MakeFunc(TypeOf(f), func(in []Value) []Value { return in }) ValueOf(&f).Elem().Set(fv) @@ -1452,8 +1454,6 @@ } } -*/ - type Point struct { x, y int } diff -r 024105249263 libgo/go/reflect/makefunc.go --- a/libgo/go/reflect/makefunc.go Tue Sep 24 20:26:38 2013 -0700 +++ b/libgo/go/reflect/makefunc.go Fri Sep 27 08:06:13 2013 -0700 @@ -7,6 +7,7 @@ package reflect import ( + "runtime" "unsafe" ) @@ -45,14 +46,33 @@ panic("reflect: call of MakeFunc with non-Func type") } + switch runtime.GOARCH { + case "amd64": + default: + panic("reflect.MakeFunc not implemented for " + runtime.GOARCH) + } + t := typ.common() ftyp := (*funcType)(unsafe.Pointer(t)) - _, _ = t, ftyp + // Indirect Go func value (dummy) to obtain + // actual code address. (A Go func value is a pointer + // to a C function pointer. http://golang.org/s/go11func.) + dummy := makeFuncStub + code := **(**uintptr)(unsafe.Pointer(&dummy)) - panic("reflect MakeFunc not implemented") + impl := &makeFuncImpl{code: code, typ: ftyp, fn: fn} + + return Value{t, unsafe.Pointer(impl), flag(Func) << flagKindShift} } +// makeFuncStub is an assembly function that is the code half of +// the function returned from MakeFunc. It expects a *callReflectFunc +// as its context register, and its job is to invoke callReflect(ctxt, frame) +// where ctxt is the context register and frame is a pointer to the first +// word in the passed-in argument frame. +func makeFuncStub() + // makeMethodValue converts v from the rcvr+method index representation // of a method value to an actual method func value, which is // basically the receiver value with a special bit set, into a true diff -r 024105249263 libgo/go/reflect/makefunc_amd64.S --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/libgo/go/reflect/makefunc_amd64.S Fri Sep 27 08:06:13 2013 -0700 @@ -0,0 +1,107 @@ +# Copyright 2013 The Go Authors. All rights reserved. +# Use of this source code is governed by a BSD-style +# license that can be found in the LICENSE file. + +# MakeFunc amd64 assembly code. + + .global reflect.makeFuncStub + +#ifdef __ELF__ + .type reflect.makeFuncStub,@function +#endif + +reflect.makeFuncStub: + .cfi_startproc + + # Store all the parameter registers in a struct that looks + # like: + # struct { + # rax uint64 // 0x0 + # rdi uint64 // 0x8 + # rsi uint64 // 0x10 + # rdx uint64 // 0x18 + # rcx uint64 // 0x20 + # r8 uint64 // 0x28 + # r9 uint64 // 0x30 + # rsp uint64 // 0x38 Pointer to arguments on stack. + # xmm0 [2]uint64 // 0x40 + # xmm1 [2]uint64 // 0x50 + # xmm2 [2]uint64 // 0x60 + # xmm3 [2]uint64 // 0x70 + # xmm4 [2]uint64 // 0x80 + # xmm5 [2]uint64 // 0x90 + # xmm6 [2]uint64 // 0xa0 + # xmm7 [2]uint64 // 0xb0 + # }; + + pushq %rbp + .cfi_def_cfa_offset 16 + .cfi_offset %rbp, -16 + movq %rsp, %rbp + .cfi_def_cfa_register %rbp + + subq $0xc0, %rsp # Space for struct on stack. + + movq %rax, 0x0(%rsp) + movq %rdi, 0x8(%rsp) + movq %rsi, 0x10(%rsp) + movq %rdx, 0x18(%rsp) + movq %rcx, 0x20(%rsp) + movq %r8, 0x28(%rsp) + movq %r9, 0x30(%rsp) + leaq 16(%rbp), %rax + movq %rax, 0x38(%rsp) + movdqa %xmm0, 0x40(%rsp) + movdqa %xmm1, 0x50(%rsp) + movdqa %xmm2, 0x60(%rsp) + movdqa %xmm3, 0x70(%rsp) + movdqa %xmm4, 0x80(%rsp) + movdqa %xmm5, 0x90(%rsp) + movdqa %xmm6, 0xa0(%rsp) + movdqa %xmm7, 0xb0(%rsp) + + # Get function type. +#ifdef __PIC__ + call __go_get_closure@PLT +#else + call __go_get_closure +#endif + movq %rax, %rsi + + movq %rsp, %rdi + +#ifdef __PIC__ + call reflect.MakeFuncStubGo@PLT +#else + call reflect.MakeFuncStubGo +#endif + + # The structure will be updated with any return values. Load + # all possible return registers before returning to the caller. + + movq 0x0(%rsp), %rax + movq 0x18(%rsp), %rdx + movq 0x8(%rsp), %rdi + movq 0x10(%rsp), %rsi + movdqa 0x40(%rsp), %xmm0 + movdqa 0x50(%rsp), %xmm1 + + # long double values are returned on the floating point stack, + # but we don't worry about that since Go doesn't have a long + # double type. + + leave + .cfi_def_cfa %rsp, 8 + + ret + + .cfi_endproc +#ifdef __ELF__ + .size reflect.makeFuncStub, . - reflect.makeFuncStub +#endif + +#ifdef __ELF__ + .section .note.GNU-stack,"",@progbits + .section .note.GNU-split-stack,"",@progbits + .section .note.GNU-no-split-stack,"",@progbits +#endif diff -r 024105249263 libgo/go/reflect/makefunc_dummy.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/libgo/go/reflect/makefunc_dummy.c Fri Sep 27 08:06:13 2013 -0700 @@ -0,0 +1,12 @@ +// Copyright 2013 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +// +build !amd64 + +// Dummy function for processors without makefunc support. + +void makeFuncStub () __asm__ ("reflect.makeFuncStub"); +void makeFuncStub () +{ +} diff -r 024105249263 libgo/go/reflect/makefuncgo_amd64.go --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/libgo/go/reflect/makefuncgo_amd64.go Fri Sep 27 08:06:13 2013 -0700 @@ -0,0 +1,487 @@ +// Copyright 2013 The Go Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + +// MakeFunc amd64 implementation. + +package reflect + +import "unsafe" + +// The assembler stub will pass a pointer to this structure. +// This will come in holding all the registers that might hold +// function parameters. On return we will set the registers that +// might hold result values. +type amd64Regs struct { + rax uint64 + rdi uint64 + rsi uint64 + rdx uint64 + rcx uint64 + r8 uint64 + r9 uint64 + rsp uint64 + xmm0 [2]uint64 + xmm1 [2]uint64 + xmm2 [2]uint64 + xmm3 [2]uint64 + xmm4 [2]uint64 + xmm5 [2]uint64 + xmm6 [2]uint64 + xmm7 [2]uint64 +} + +// Argument classifications. The amd64 ELF ABI uses several more, but +// these are the only ones that arise for Go types. +type amd64Class int + +const ( + amd64Integer amd64Class = iota + amd64SSE + amd64NoClass + amd64Memory +) + +// amd64Classify returns the one or two register classes needed to +// pass the value of type. Go types never need more than two +// registers. amd64Memory means the value is stored in memory. +// amd64NoClass means the register is not used. +func amd64Classify(typ *rtype) (amd64Class, amd64Class) { + switch typ.Kind() { + default: + panic("internal error--unknown kind in amd64Classify") + + case Bool, Int, Int8, Int16, Int32, Int64, + Uint, Uint8, Uint16, Uint32, Uint64, + Uintptr, Chan, Func, Map, Ptr, UnsafePointer: + + return amd64Integer, amd64NoClass + + case Float32, Float64, Complex64: + return amd64SSE, amd64NoClass + + case Complex128: + return amd64SSE, amd64SSE + + case Array: + if typ.size == 0 { + return amd64NoClass, amd64NoClass + } else if typ.size > 16 { + return amd64Memory, amd64NoClass + } + atyp := (*arrayType)(unsafe.Pointer(typ)) + eclass1, eclass2 := amd64Classify(atyp.elem) + if eclass1 == amd64Memory { + return amd64Memory, amd64NoClass + } + if eclass2 == amd64NoClass && typ.size > 8 { + eclass2 = eclass1 + } + return eclass1, eclass2 + + case Interface: + return amd64Integer, amd64Integer + + case Slice: + return amd64Memory, amd64NoClass + + case String: + return amd64Integer, amd64Integer + + case Struct: + if typ.size == 0 { + return amd64NoClass, amd64NoClass + } else if typ.size > 16 { + return amd64Memory, amd64NoClass + } + var first, second amd64Class + f := amd64NoClass + onFirst := true + styp := (*structType)(unsafe.Pointer(typ)) + for _, field := range styp.fields { + if onFirst && field.offset >= 8 { + first = f + f = amd64NoClass + onFirst = false + } + fclass1, fclass2 := amd64Classify(field.typ) + f = amd64MergeClasses(f, fclass1) + if fclass2 != amd64NoClass { + if !onFirst { + panic("amd64Classify inconsistent") + } + first = f + f = fclass2 + onFirst = false + } + } + if onFirst { + first = f + second = amd64NoClass + } else { + second = f + } + if first == amd64Memory || second == amd64Memory { + return amd64Memory, amd64NoClass + } + return first, second + } +} + +// amd64MergeClasses merges two register classes as described in the +// amd64 ELF ABI. +func amd64MergeClasses(c1, c2 amd64Class) amd64Class { + switch { + case c1 == c2: + return c1 + case c1 == amd64NoClass: + return c2 + case c2 == amd64NoClass: + return c1 + case c1 == amd64Memory || c2 == amd64Memory: + return amd64Memory + case c1 == amd64Integer || c2 == amd64Integer: + return amd64Integer + default: + return amd64SSE + } +} + +// MakeFuncStubGo implements the amd64 calling convention for +// MakeFunc. This should not be called. It is exported so that +// assembly code can call it. + +func MakeFuncStubGo(regs *amd64Regs, c *makeFuncImpl) { + ftyp := c.typ + + // See if the result requires a struct. If it does, the first + // parameter is a pointer to the struct. + var ret1, ret2 amd64Class + switch len(ftyp.out) { + case 0: + ret1, ret2 = amd64NoClass, amd64NoClass + case 1: + ret1, ret2 = amd64Classify(ftyp.out[0]) + default: + off := uintptr(0) + f := amd64NoClass + onFirst := true + for _, rt := range ftyp.out { + off = align(off, uintptr(rt.fieldAlign)) + + if onFirst && off >= 8 { + ret1 = f + f = amd64NoClass + onFirst = false + } + + off += rt.size + if off > 16 { + break + } + + fclass1, fclass2 := amd64Classify(rt) + f = amd64MergeClasses(f, fclass1) + if fclass2 != amd64NoClass { + if !onFirst { + panic("amd64Classify inconsistent") + } + ret1 = f + f = fclass2 + onFirst = false + } + } + if off > 16 { + ret1, ret2 = amd64Memory, amd64NoClass + } else { + if onFirst { + ret1, ret2 = f, amd64NoClass + } else { + ret2 = f + } + } + if ret1 == amd64Memory || ret2 == amd64Memory { + ret1, ret2 = amd64Memory, amd64NoClass + } + } + + in := make([]Value, 0, len(ftyp.in)) + intreg := 0 + ssereg := 0 + ap := uintptr(regs.rsp) + + maxIntregs := 6 // When we support Windows, this would be 4. + maxSSEregs := 8 + + if ret1 == amd64Memory { + // We are returning a value in memory, which means + // that the first argument is a hidden parameter + // pointing to that return area. + intreg++ + } + +argloop: + for _, rt := range ftyp.in { + c1, c2 := amd64Classify(rt) + + fl := flag(rt.Kind()) << flagKindShift + if c2 == amd64NoClass { + + // Argument is passed in a single register or + // in memory. + + switch c1 { + case amd64NoClass: + v := Value{rt, nil, fl | flagIndir} + in = append(in, v) + continue argloop + case amd64Integer: + if intreg < maxIntregs { + reg := amd64IntregVal(regs, intreg) + iw := unsafe.Pointer(reg) + if k := rt.Kind(); k != Ptr && k != UnsafePointer { + iw = unsafe.Pointer(®) + fl |= flagIndir + } + v := Value{rt, iw, fl} + in = append(in, v) + intreg++ + continue argloop + } + case amd64SSE: + if ssereg < maxSSEregs { + reg := amd64SSEregVal(regs, ssereg) + v := Value{rt, unsafe.Pointer(®), fl | flagIndir} + in = append(in, v) + ssereg++ + continue argloop + } + } + + in, ap = amd64Memarg(in, ap, rt) + continue argloop + } + + // Argument is passed in two registers. + + nintregs := 0 + nsseregs := 0 + switch c1 { + case amd64Integer: + nintregs++ + case amd64SSE: + nsseregs++ + default: + panic("inconsistent") + } + switch c2 { + case amd64Integer: + nintregs++ + case amd64SSE: + nsseregs++ + default: + panic("inconsistent") + } + + // If the whole argument does not fit in registers, it + // is passed in memory. + + if intreg+nintregs > maxIntregs || ssereg+nsseregs > maxSSEregs { + in, ap = amd64Memarg(in, ap, rt) + continue argloop + } + + var word1, word2 uintptr + switch c1 { + case amd64Integer: + word1 = amd64IntregVal(regs, intreg) + intreg++ + case amd64SSE: + word1 = amd64SSEregVal(regs, ssereg) + ssereg++ + } + switch c2 { + case amd64Integer: + word2 = amd64IntregVal(regs, intreg) + intreg++ + case amd64SSE: + word2 = amd64SSEregVal(regs, ssereg) + ssereg++ + } + + p := unsafe_New(rt) + *(*uintptr)(p) = word1 + *(*uintptr)(unsafe.Pointer(uintptr(p) + ptrSize)) = word2 + v := Value{rt, p, fl | flagIndir} + in = append(in, v) + } + + // All the real arguments have been found and turned into + // Value's. Call the real function. + + out := c.fn(in) + + if len(out) != len(ftyp.out) { + panic("reflect: wrong return count from function created by MakeFunc") + } + + for i, typ := range ftyp.out { + v := out[i] + if v.typ != typ { + panic("reflect: function created by MakeFunc using " + funcName(c.fn) + + " returned wrong type: have " + + out[i].typ.String() + " for " + typ.String()) + } + if v.flag&flagRO != 0 { + panic("reflect: function created by MakeFunc using " + funcName(c.fn) + + " returned value obtained from unexported field") + } + } + + if ret1 == amd64NoClass { + return + } + + if ret1 == amd64Memory { + // The address of the memory area was passed as a + // hidden parameter in %rdi. + ptr := unsafe.Pointer(uintptr(regs.rdi)) + off := uintptr(0) + for i, typ := range ftyp.out { + v := out[i] + off = align(off, uintptr(typ.fieldAlign)) + addr := unsafe.Pointer(uintptr(ptr) + off) + if v.flag&flagIndir == 0 && (v.kind() == Ptr || v.kind() == UnsafePointer) { + storeIword(addr, iword(v.val), typ.size) + } else { + memmove(addr, v.val, typ.size) + } + off += typ.size + } + return + } + + if len(out) == 1 && ret2 == amd64NoClass { + v := out[0] + w := v.iword() + if v.Kind() != Ptr && v.Kind() != UnsafePointer { + w = loadIword(unsafe.Pointer(w), v.typ.size) + } + switch ret1 { + case amd64Integer: + regs.rax = uint64(uintptr(w)) + case amd64SSE: + regs.xmm0[0] = uint64(uintptr(w)) + regs.xmm0[1] = 0 + default: + panic("inconsistency") + } + return + } + + var buf [2]unsafe.Pointer + ptr := unsafe.Pointer(&buf[0]) + off := uintptr(0) + for i, typ := range ftyp.out { + v := out[i] + off = align(off, uintptr(typ.fieldAlign)) + addr := unsafe.Pointer(uintptr(ptr) + off) + if v.flag&flagIndir == 0 && (v.kind() == Ptr || v.kind() == UnsafePointer) { + storeIword(addr, iword(v.val), typ.size) + } else { + memmove(addr, v.val, typ.size) + } + off += uintptr(typ.size) + } + + switch ret1 { + case amd64Integer: + regs.rax = *(*uint64)(unsafe.Pointer(&buf[0])) + case amd64SSE: + regs.xmm0[0] = *(*uint64)(unsafe.Pointer(&buf[0])) + regs.xmm0[1] = 0 + default: + panic("inconsistency") + } + + switch ret2 { + case amd64Integer: + reg := *(*uint64)(unsafe.Pointer(&buf[1])) + if ret1 == amd64Integer { + regs.rdx = reg + } else { + regs.rax = reg + } + case amd64SSE: + reg := *(*uint64)(unsafe.Pointer(&buf[1])) + if ret1 == amd64Integer { + regs.xmm0[0] = reg + regs.xmm0[1] = 0 + } else { + regs.xmm1[0] = reg + regs.xmm1[1] = 0 + } + case amd64NoClass: + default: + panic("inconsistency") + } +} + +// The amd64Memarg function adds an argument passed in memory. +func amd64Memarg(in []Value, ap uintptr, rt *rtype) ([]Value, uintptr) { + ap = align(ap, ptrSize) + ap = align(ap, uintptr(rt.align)) + p := Value{rt, unsafe.Pointer(ap), flag(rt.Kind()< 2 } -// callReflect is the call implementation used by a function -// returned by MakeFunc. In many ways it is the opposite of the -// method Value.call above. The method above converts a call using Values -// into a call of a function with a concrete argument frame, while -// callReflect converts a call of a function with a concrete argument -// frame into a call using Values. -// It is in this file so that it can be next to the call method above. -// The remainder of the MakeFunc implementation is in makefunc.go. -func callReflect(ctxt *makeFuncImpl, frame unsafe.Pointer) { - ftyp := ctxt.typ - f := ctxt.fn - - // Copy argument frame into Values. - ptr := frame - off := uintptr(0) - in := make([]Value, 0, len(ftyp.in)) - for _, arg := range ftyp.in { - typ := arg - off += -off & uintptr(typ.align-1) - v := Value{typ, nil, flag(typ.Kind()) << flagKindShift} - if typ.size <= ptrSize { - // value fits in word. - v.val = unsafe.Pointer(loadIword(unsafe.Pointer(uintptr(ptr)+off), typ.size)) - } else { - // value does not fit in word. - // Must make a copy, because f might keep a reference to it, - // and we cannot let f keep a reference to the stack frame - // after this function returns, not even a read-only reference. - v.val = unsafe_New(typ) - memmove(v.val, unsafe.Pointer(uintptr(ptr)+off), typ.size) - v.flag |= flagIndir - } - in = append(in, v) - off += typ.size - } - - // Call underlying function. - out := f(in) - if len(out) != len(ftyp.out) { - panic("reflect: wrong return count from function created by MakeFunc") - } - - // Copy results back into argument frame. - if len(ftyp.out) > 0 { - off += -off & (ptrSize - 1) - for i, arg := range ftyp.out { - typ := arg - v := out[i] - if v.typ != typ { - panic("reflect: function created by MakeFunc using " + funcName(f) + - " returned wrong type: have " + - out[i].typ.String() + " for " + typ.String()) - } - if v.flag&flagRO != 0 { - panic("reflect: function created by MakeFunc using " + funcName(f) + - " returned value obtained from unexported field") - } - off += -off & uintptr(typ.align-1) - addr := unsafe.Pointer(uintptr(ptr) + off) - if v.flag&flagIndir == 0 { - storeIword(addr, iword(v.val), typ.size) - } else { - memmove(addr, v.val, typ.size) - } - off += typ.size - } - } -} - // methodReceiver returns information about the receiver // described by v. The Value v may or may not have the // flagMethod bit set, so the kind cached in v.flag should